Repository navigation
Parse protobuf battle journals and keep the JSON journal shape - #392
Conversation
Since 2026-11-19 the game server answers POST /journals/get with protobuf
instead of JSON: a ServiceResponse whose entity group 221 (BattleReport)
holds a JournalResponse. The JSON parse failed, so no battle journal
reached any sync target.
The mod now converts the JournalResponse into the journal object the JSON
server sent: same keys, nesting and JSON types, so sync consumers keep
working unchanged. Keys the server no longer sends are left out rather
than invented, and converted journals carry "journal_src":"pb1". Bodies
that start with '{' still take the JSON path.
The battle report messages are added to the models proto as new messages
only; existing messages are untouched. Participant lookup now reads
deployed_fleets as the object it is in journals.
Validated with a synthetic unit test and against 20 captured journals of
single-fleet battles. Armada, PvP and station battles are untested.
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
|
Independent validation from a second setup. I was chasing the same break when this landed, so I checked it against what I had. It works for me; there are two small notes below. Wire and schema
Converter output
Tests
In game (1.1.10 release layout)
The validation and this write-up were done with help from Claude Code. |
|
Thanks for the independent check, and for the live armada.
|
What changed and why
Since 2026-11-19 the game server answers
POST /journals/getwith protobuf instead of JSON. The body is aServiceResponsewhose entity group 221 (TYPE_BATTLEREPORT) holds aJournalResponse.FetchBattlestill parsedthe body as JSON, so every journal failed and battle log sync stopped for all targets. Fixes #391.
The mod now converts the
JournalResponseback into the journal object the JSON server sent. It keeps the same keynames, nesting and JSON types, so sync consumers and the
battlelogrecord stay unchanged. Bodies whose firstnon-whitespace byte is
{still take the JSON path.src/mod/sync/journal.h,journal.cc(new):IsJsonBody()andJournalFromProtobuf(). They depend only on thegenerated protobuf code, nlohmann::json and the standard library. Conversion rules:
1.1, not1.100000023841858), throughto_charsand the JSON parser,as
ShortNumberdoes.battle_timeisYYYY-MM-DDTHH:MM:SSUTC.battle_logentries keep their number text, so5.0stays a float.bridge_officersis the 10-slot crew array with nulls,{}for an NPC side andnullfor an armada.src/mod/sync/battlelogs.cc:FetchBattlepicks the path from the first byte and logs one error with the battleid, body size and reason when a protobuf journal does not convert. As before, it then retries.
Participantsnowreads
deployed_fleetsas an object keyed by fleet id, as journals have always sent it. The oldis_array()check skipped it, so only
initiator_id/target_idwere resolved. Empty uids are skipped.src/mod/game/proto/Digit.PrimeServer.Models.proto: 10 new messages (JournalResponse,JournalFleetData,JournalFleetOfficerEntry,BridgeCrewOfficer,CargoProto,JournalChestDrop,JournalContextData,JournalContextDataInner,JournalMultitargetArmadaInfo,JournalGalacticAnomalyData). They were recovered fromthe client's message descriptors and checked against live responses. These are pure additions: existing messages,
including
DeployedFleetandJournalHeaderProto, are untouched.zig/deps/protobuf_roots.txt:JournalResponseandServiceResponse.tests/sync_logic.cc,zig/tests.zig: aprotobuf journalstest intest-sync.CONTRIBUTING.md:sync/journal.ccin the sync ownership table.Converted journals carry
"journal_src":"pb1"inside the journal object, so a consumer can tell them apart.Keys the server no longer sends are left out, never set to 0 or invented. They return automatically if the server
sends the backing field again:
num_references.loot_roll_key,loot_roll_type.params:chest_name,items,chest_item_<id>_amount/_type,source,id.faction_id,active_officer_slots; andfleet_idon an armada side, whose old value isnot derivable.
battle_start_time,state,is_activeand the otheris_*flags;node_id,course_id,latest_course_vector_x/_y,state_updated_at;internal_stats,mining_amount_bonus_stats,node_address;active_buffs[].attributes.xp_gainedandwarp_datastaynull, as in every JSON journal. Values that were server doubles are now float32on the wire. Integral values are exact; fractional stats and ratings above 2^24 can differ in the last digits.
Validation
zig build mods build-tests -Dtarget=x86_64-windows-msvc,zig build test-sync test-mod test-codegen, the fullReleaseFastDLL build, andzig build fmt.protobuf journalstest builds a synthetic player-vs-NPCServiceResponsewith the protobuf API. Itcompares the result with an exact expected journal, number kinds included. It also checks that a JSON body is
detected, and that empty, garbage, truncated, wrong-group and id-less bodies are refused.
integers digit for digit and all keys and types. The only differences are float32 values that sit exactly halfway
between two shortest forms, where
to_charsrounds half to even. A value-level self-check of 1,430 assertionsagainst the decoded wire passed.
The
devbuild is compile-, unit- and parity-tested only, not run in game.build-wincurrently fails on the runner's newer MSVC toolset, withSTL1000in every unit includingdependencies. This is unrelated to this change.
Settings and platforms
battlelogs).the same rules. A single-ship armada (type 8) was validated independently, see the comments. An armada
keeps
deployed_fleetskeyed by fleet id,bridge_officersisnulland the side'sfleet_idisleft out. A side without a fleet leaves out
deployed_fleet.journal_srcmarker. Match armada participants by hull or ship id, not by array position. Measurement fields can differ in
number kind from the JSON journals: ratings, hit points and shields are int64 on the wire,
coordsand the*_level_percentagesare float32. Ids andbattle_logmarkers stay integers.from_chars, which Apple's libc++ lacksbefore macOS 26.
🤖 Generated with Claude Code