MCPcopy
hub / github.com/huggingface/transformers / split_code_into_blocks

Function split_code_into_blocks

utils/check_copies.py:288–393  ·  view source on GitHub ↗

Split the class/func block starting at `start_index` in a source code (defined by `lines`) into *inner blocks*. The block's header is included as the first element. The contiguous regions (without empty lines) that are not inside any inner block are included as blocks. The contiguous r

(
    lines: list[str], start_index: int, end_index: int, indent: int, backtrace: bool = False
)

Source from the content-addressed store, hash-verified

286
287
288def split_code_into_blocks(
289 lines: list[str], start_index: int, end_index: int, indent: int, backtrace: bool = False
290) -> list[tuple[str, int, int]]:
291 """
292 Split the class/func block starting at `start_index` in a source code (defined by `lines`) into *inner blocks*.
293
294 The block's header is included as the first element. The contiguous regions (without empty lines) that are not
295 inside any inner block are included as blocks. The contiguous regions of empty lines that are not inside any inner
296 block are also included as (dummy) blocks.
297
298 Args:
299 lines (`List[str]`):
300 The source code, represented by a list of lines.
301 start_index (`int`):
302 The starting index of the target class/func block.
303 end_index (`int`):
304 The ending index of the target class/func block.
305 indent (`int`):
306 The indent of the class/func body.
307 backtrace (`bool`, *optional*, defaults to `False`):
308 Whether or not to include the lines before the inner class/func block's header (e.g. comments, decorators,
309 etc.) until an empty line is encountered.
310
311 Returns:
312 `List[Tuple[str, int, int]]`: A list of elements with the form `(block_name, start_index, end_index)`.
313 """
314 splits = []
315 # `indent - 4` is the indent level of the target class/func header
316 try:
317 target_block_name = re.search(
318 rf"^{' ' * (indent - 4)}((class|def)\s+\S+)(\(|\:)", lines[start_index]
319 ).groups()[0]
320 except Exception:
321 start_context = min(start_index - 10, 0)
322 end_context = min(end_index + 10, len(lines))
323 raise ValueError(
324 f"Tried to split a class or function. It did not work. Error comes from line {start_index}: \n```\n"
325 + "".join(lines[start_context:end_context])
326 + "```\n"
327 )
328
329 # from now on, the `block` means inner blocks unless explicitly specified
330 indent_str = " " * indent
331 block_without_name_idx = 0
332 empty_block_idx = 0
333
334 # Find the lines for the definition header
335 index = start_index
336 if "(" in lines[start_index] and "):" not in lines[start_index] in lines[start_index]:
337 while index < end_index:
338 if _is_definition_header_ending_line(lines[index]):
339 break
340 index += 1
341
342 # the first line outside the definition header
343 index += 1
344 splits.append((target_block_name, start_index, index))
345

Callers 2

find_code_and_splitsFunction · 0.85
is_copy_consistentFunction · 0.85

Calls 3

find_block_endFunction · 0.85
joinMethod · 0.80

Tested by

no test coverage detected