MCPcopy Index your code
hub / github.com/python/cpython / _iter_source

Function _iter_source

Tools/c-analyzer/c_parser/parser/__init__.py:169–236  ·  view source on GitHub ↗
(lines, *, maxtext=11_000, maxlines=200, showtext=False)

Source from the content-addressed store, hash-verified

167# are covered elsewhere (MAX_SIZES in cpython/_parser.py).
168
169def _iter_source(lines, *, maxtext=11_000, maxlines=200, showtext=False):
170 maxtext = maxtext if maxtext and maxtext > 0 else None
171 maxlines = maxlines if maxlines and maxlines > 0 else None
172 filestack = []
173 allinfo = {}
174 # "lines" should be (fileinfo, data), as produced by the preprocessor code.
175 for fileinfo, line in lines:
176 if fileinfo.filename in filestack:
177 while fileinfo.filename != filestack[-1]:
178 filename = filestack.pop()
179 del allinfo[filename]
180 filename = fileinfo.filename
181 srcinfo = allinfo[filename]
182 else:
183 filename = fileinfo.filename
184 srcinfo = SourceInfo(filename)
185 filestack.append(filename)
186 allinfo[filename] = srcinfo
187
188 _logger.debug(f'-> {line}')
189 srcinfo._add_line(line, fileinfo.lno)
190 if srcinfo.too_much(maxtext, maxlines):
191 break
192 while srcinfo._used():
193 yield srcinfo
194 if showtext:
195 _logger.debug(f'=> {srcinfo.text}')
196 else:
197 if not filestack:
198 srcinfo = SourceInfo('???')
199 else:
200 filename = filestack[-1]
201 srcinfo = allinfo[filename]
202 while srcinfo._used():
203 yield srcinfo
204 if showtext:
205 _logger.debug(f'=> {srcinfo.text}')
206 yield srcinfo
207 if showtext:
208 _logger.debug(f'=> {srcinfo.text}')
209 if not srcinfo._ready:
210 return
211 # At this point either the file ended prematurely
212 # or there's "too much" text.
213 filename, lno_from, lno_to = srcinfo.filename, srcinfo.start, srcinfo.end
214 text = srcinfo.text
215 if len(text) > 500:
216 text = text[:500] + '...'
217
218 if srcinfo.too_much_text(maxtext):
219 msg = f'''
220 too much text, try to increase MAX_SIZES[MAXTEXT] in cpython/_parser.py
221 {filename} starting at line {lno_from} to {lno_to}
222 has code with length {len(srcinfo.text)} greater than {maxtext}:
223 {text}
224 '''
225 raise RuntimeError(textwrap.dedent(msg))
226

Callers 1

_parseFunction · 0.85

Calls 10

_add_lineMethod · 0.95
too_muchMethod · 0.95
_usedMethod · 0.95
too_much_textMethod · 0.95
too_many_linesMethod · 0.95
SourceInfoClass · 0.85
popMethod · 0.45
appendMethod · 0.45
debugMethod · 0.45
dedentMethod · 0.45

Tested by

no test coverage detected

Used in the wild real call sites across dependent graphs

searching dependent graphs…