MCPcopy Create free account
hub / github.com/segmentio/encoding / NewTokenizer

Function NewTokenizer

json/token.go:75–80  ·  view source on GitHub ↗

NewTokenizer constructs a new Tokenizer which reads its json input from b.

(b []byte)

Source from the content-addressed store, hash-verified

73
74// NewTokenizer constructs a new Tokenizer which reads its json input from b.
75func NewTokenizer(b []byte) *Tokenizer {
76 return &Tokenizer{
77 json: b,
78 decoder: decoder{flags: internalParseFlags(b)},
79 }
80}
81
82// Reset erases the state of t and re-initializes it with the json input from b.
83func (t *Tokenizer) Reset(b []byte) {

Callers 3

tokenizeFunction · 0.85
BenchmarkTokenizerFunction · 0.85

Calls 1

internalParseFlagsFunction · 0.85

Tested by 3

tokenizeFunction · 0.68
BenchmarkTokenizerFunction · 0.68

Used in the wild real call sites across dependent graphs

searching dependent graphs…