Generator that consumes a line iterated iterable and strips out the multiple (or multi-character) comments from lines. This is a pre-processing step to achieve feature parity with loadtxt (we assume that this feature is a niche feature).
(iterable, comments, encoding)
| 837 | |
| 838 | |
| 839 | def _preprocess_comments(iterable, comments, encoding): |
| 840 | """ |
| 841 | Generator that consumes a line iterated iterable and strips out the |
| 842 | multiple (or multi-character) comments from lines. |
| 843 | This is a pre-processing step to achieve feature parity with loadtxt |
| 844 | (we assume that this feature is a niche feature). |
| 845 | """ |
| 846 | for line in iterable: |
| 847 | if isinstance(line, bytes): |
| 848 | # Need to handle conversion here, or the splitting would fail |
| 849 | line = line.decode(encoding) |
| 850 | |
| 851 | for c in comments: |
| 852 | line = line.split(c, 1)[0] |
| 853 | |
| 854 | yield line |
| 855 | |
| 856 | |
| 857 | # The number of rows we read in one go if confronted with a parametric dtype |