MCPcopy Create free account
hub / github.com/ChatLab/ChatLab / applyDataCleaning

Function applyDataCleaning

packages/node-runtime/src/ai/preprocessor/pipeline.ts:85–95  ·  view source on GitHub ↗
(messages: T[])

Source from the content-addressed store, hash-verified

83// ==================== 策略实现 ====================
84
85function applyDataCleaning<T extends PreprocessableMessage>(messages: T[]): { messages: T[]; changed: number } {
86 let changed = 0
87 const result = messages.map((msg) => {
88 if (!msg.content) return msg
89 const cleaned = cleanXmlContent(msg.content)
90 if (cleaned === msg.content) return msg
91 changed++
92 return { ...msg, content: cleaned }
93 })
94 return { messages: result, changed }
95}
96
97const XML_START = /^<\?xml\s|^<msg[\s>]/
98

Callers 1

preprocessMessagesFunction · 0.85

Calls 1

cleanXmlContentFunction · 0.85

Tested by

no test coverage detected