Generator that consumes a line iterated iterable and strips out the multiple (or multi-character) comments from lines. This is a pre-processing step to achieve feature parity with loadtxt (we assume that this feature is a nieche feature).
(iterable, comments, encoding)
| 794 | |
| 795 | |
| 796 | def _preprocess_comments(iterable, comments, encoding): |
| 797 | """ |
| 798 | Generator that consumes a line iterated iterable and strips out the |
| 799 | multiple (or multi-character) comments from lines. |
| 800 | This is a pre-processing step to achieve feature parity with loadtxt |
| 801 | (we assume that this feature is a nieche feature). |
| 802 | """ |
| 803 | for line in iterable: |
| 804 | if isinstance(line, bytes): |
| 805 | # Need to handle conversion here, or the splitting would fail |
| 806 | line = line.decode(encoding) |
| 807 | |
| 808 | for c in comments: |
| 809 | line = line.split(c, 1)[0] |
| 810 | |
| 811 | yield line |
| 812 | |
| 813 | |
| 814 | # The number of rows we read in one go if confronted with a parametric dtype |