BUG: Read SAS fails · Issue #12659 · pandas-dev/pandas (original) (raw)
cc: @kshedden
Code Sample, a copy-pastable example if possible
Do you want a PR with this test case?
Expected Output
pandas reads the file without crashing :)
I tracked the problem to _process_columntext_subheader. It seems like there is an encoding issue...
You might need to apply PR #12658 first to see this issue.
FWIW, here is an AWFUL HACK to make pandas read the file "almost" correctly (I get column names as bytes instead of str in this case).
@@ -524,17 +524,17 @@ class SAS7BDATReader(BaseIterator):
offset += self._int_length
text_block_size = self._read_int(offset, _text_block_size_length)
buf = self._read_bytes(offset, text_block_size)self.column_names_strings.append(buf[0:text_block_size].rstrip(b"\x00 ").decode())
self.column_names_strings.append(buf) if len(self.column_names_strings) == 1: column_name = self.column_names_strings[0] compression_literal = "" for cl in _compression_literals:cl = bytes(cl, 'ascii') if cl in column_name: compression_literal = cl self.compression = compression_literal offset -= self._int_length
output of pd.show_versions()
commit: None
python: 3.5.1.final.0
python-bits: 64
OS: Windows
OS-release: 10
machine: AMD64
processor: Intel64 Family 6 Model 69 Stepping 1, GenuineIntel
byteorder: little
LC_ALL: None
LANG: None
pandas: 0.18.0
nose: 1.3.7
pip: 8.1.0
setuptools: 20.2.2
Cython: None
numpy: 1.10.4
scipy: None
statsmodels: None
xarray: None
IPython: 4.1.2
sphinx: 1.3.1
patsy: None
dateutil: 2.4.2
pytz: 2015.7
blosc: None
bottleneck: 1.0.0
tables: 3.2.2
numexpr: 2.4.6
matplotlib: 1.5.1
openpyxl: 2.3.2
xlrd: 0.9.4
xlwt: None
xlsxwriter: 0.8.4
lxml: 3.5.0
bs4: None
html5lib: None
httplib2: None
apiclient: None
sqlalchemy: None
pymysql: None
psycopg2: None
jinja2: 2.8
boto: None