Repository navigation
Conversation
- Clarify that Kafka input accepts skipped_line but does not apply it. - Recommend preprocessing the topic or using FILE/HDFS for line filtering. - Update both English and Chinese guides.
94c6d6f to
0536c41
Compare
bitflicker64
left a comment
There was a problem hiding this comment.
Blocking: no. Summary: The note matches KafkaReader, which hands each message to the parser and never reads skippedLine, but both pages already state this in the Kafka skipped_line bullet, so the note duplicates it above the page intro.
| search_boost: 1.6 | ||
| description: "Bulk import graph data into HugeGraph with Loader from files, HDFS, relational databases, Kafka, and other HugeGraph graphs." | ||
| --- | ||
| > **Kafka and `skipped_line`:** HugeGraph Loader accepts `skipped_line` for |
There was a problem hiding this comment.
Minor: the skipped_line bullet under 3.3.2.4 Kafka input source already documents this limitation on both pages, and this note renders above the page intro instead of below the Kafka options. Remove the note from the English and Chinese pages, or move the FILE/HDFS suggestion into the existing skipped_line bullet.
imbajin
left a comment
There was a problem hiding this comment.
Score: 8.0/10 · Confidence: medium
Blocking: no
Summary
The existing inline concern remains: the note duplicates the Kafka option description and sits above the introduction. Consolidating the FILE/HDFS suggestion in that option would keep the guidance together.
Evidence
KafkaSource declares skipped_line and KafkaReader passes each message to the parser without applying it; FileLineFetcher applies the file filter. Both language pages already document the Kafka limitation in their Kafka options.
Validation notes
Source review only; no live Kafka/HDFS test or Hugo build was run.
Purpose of the PR
skipped_line, but the loader does not apply it to Kafka messages.Main Changes
FILE/HDFSinput source when line filtering is required.skipped_linedescription for file-based input sources unchanged.Context
The loader's
KafkaSourcedeclares@JsonProperty("skipped_line"), whileKafkaReaderhas no reference to it. The current guides describeskipped_lineas if it filtered Kafka messages.Verification
scripts/hugo.sh buildwith Hugo 0.165.0 extended and--panicOnWarning: passes, 570 pages (286 EN / 284 CN).