"errors" parameter has no effect on pd.DataFrame.astype if a dictionary is passed in as the dtype argument · Issue #25905 · pandas-dev/pandas (original) (raw)

Code Sample, a copy-pastable example if possible

Given the code sample below:

data_df = pd.DataFrame([{'col_a': '1', 'col_b': '16.5%', 'col_c': 'test'}, {'col_a': '2.2', 'col_b': '15.3', 'col_c': 'another_test'} ]) type_dict = {'col_a': 'float64', 'col_b': 'float64', 'col_c': 'object} data_df = data_df.astype(dtype=type_dict, errors='ignore')

Problem description

Pandas will raise the error ValueError: could not convert string to float: '16.5%' despite the fact that the errors='ignore' parameter was passed in. This is probable due to the implementation of astype, as can be seen in pandas.core.generic.py lines 5658 - 5680

    if is_dict_like(dtype):
        if self.ndim == 1:  # i.e. Series
            if len(dtype) > 1 or self.name not in dtype:
                raise KeyError('Only the Series name can be used for '
                               'the key in Series dtype mappings.')
            new_type = dtype[self.name]
            return self.astype(new_type, copy, errors, **kwargs)
        elif self.ndim > 2:
            raise NotImplementedError(
                'astype() only accepts a dtype arg of type dict when '
                'invoked on Series and DataFrames. A single dtype must be '
                'specified when invoked on a Panel.'
            )
        for col_name in dtype.keys():
            if col_name not in self:
                raise KeyError('Only a column name can be used for the '
                               'key in a dtype mappings argument.')
        results = []
        for col_name, col in self.iteritems():
            if col_name in dtype:
                results.append(col.astype(dtype[col_name], copy=copy))
            else:
                results.append(results.append(col.copy() if copy else col))

Expected Output

One would expect the pd.DataFrame to successfully type cast, leaving the col_b as an np.object type