GGPK

Old: chuanhsing#2097 2025-02-15 15:44:33

New: chuanhsing#2097 2025-02-21 15:01:56

OldNewDifferences
1-# Version 3.26.0
1+# GGPK
22
3-## Until PoE2 0.2.0 has shipped, will start to work on PoE1 3.26.
3+## GGPK schema
44
5-<a href='#' class="js-modal-btn" data-video-id="wF4rqnPoo80"></a>
5+### Record Structures
66
7-Hey, guys, let's talk about Path of Exile 1. Now, when we announced Path of Exile 2, we promised that we would continue to create Path of Exile 1 expansions, and we do intend to keep that promise! But it's certainly been a lot harder to deliver on it than what we expected.
7+```
8+struct GGPK {
9+ uint32 Length;
10+ char[4] Tag; // ="GGPK"
11+ uint32 Version;
12+ GgpkEntry[2] Entries;
13+}
14+struct GgpkEntry {
15+ uint64 offset;
16+}
17+struct FREE {
18+ uint32 Length;
19+ char[4] Tag; // = "FREE"
20+ Byte[Length-8] Data;
21+}
22+struct PDIR {
23+ uint32 Length;
24+ char[4] Tag; // ="PDIR"
25+ uint32 NameLength;
26+ uint32 TotalEntries;
27+ byte[32] SHA256Hash;
28+ wchar_t[NameLength] Name;
29+ DirectoryEntry[TotalEntries];
30+}
31+struct DirectoryEntry {
32+ int32 entryNameHash;
33+ uint64 offset;
34+}
35+struct FILE {
36+ uint32 Length;
37+ char[4] Tag; // ="FILE"
38+ uint32 NameLength;
39+ byte[32] SHA256Hash;
40+ wchar_t[NameLength] Name;
41+ byte[reminder] Data;
42+}
43+```
844
9-Now, the trouble really started after Settlers of Kalguur in July last year.
45+<a href="https://i.imgur.com/VRBetTX.png" data-lightbox="panel"></a>
1046
11-The ideal time to have release 3.26 would have been late October.
12-before they originally scheduled release of Path of Exile 2’s Early Access, but we just didn't think we could do it.
47+### Sample Tree
48+<a href="https://i.imgur.com/EP7QRal.png" data-lightbox="panel"></a>
1349
14-It was too close to the huge circus that was the organisation around Path of Exile 2’s launch.
1550
16-But we also didn't want to immediately launch it right after Path of Exile 2 either.
51+### Reference
52+Site | Description
53+- | -
54+[Path of Go](https://github.com/oriath-net/pogo) | Golang tools for reading PoE data files
55+[LibGGPK3](https://github.com/aianlinb/LibGGPK3) | Windows GUI tool to view Content.ggpk and export
56+[libbun](https://github.com/zao/ooz) | Ooz shared library (libooz.dll/liblibooz.so) for Oodle decompression and Bun library/tool with a C API suitable for FFI/interop - reads and extracts bundle contents from a Steam game directory or a PC Standalone GGPK. Tested on x64 Windows and Debian. <br>Original Ooz code by powzix/rarten, adapted into a library.
1757
18-So that kind of ruled out December.
58+## Bundle schema
1959
20-So then we're thinking, okay, we can't really have a launch in January due to Christmas, so we're really going to have to do it in February.
60+[Bundle schema](Bundle_schema)
2161
22-And this wasn't great.
62+## Dat schema
2363
24-It would mean seven months with no leagues. But we really didn't think it was possible to do it any earlier than that.
64+```
65+struct DAT {
66+ uint32 rowCounts;
67+ Data[rowCounts] rows;
68+ REFERENCE refer;
69+}
2570
26-And we didn't really announce this exactly, but it was certainly something that we had said to quite a few people at over time.
71+struct Data {
72+ blob row;
73+}
2774
28-So the community became generally aware that that was our plan.
75+struct REFERENCE {
76+ char[8]; // 0xBB * 8
77+ blob reference;
78+}
79+```
2980
30-The issue was this, Path of Exile 2’s end game needed a lot of work.
81+### Data Export
3182
32-Having the guys on the Path of Exile1 team help out for three months on Path of Exile 2, given the 3.26, was seven months away, seemed like a no brainer.
83+The game data does not have any official API, and only possible data can be analyzed from the game files.
3384
34-And man, did Path of Exile 2 need it.
85+Almost all data is stored in .dat64 files within Content.ggpk. But they only have data, no headers. You can start with the .dat64 format, the [PoE dat schema](https://github.com/poe-tool-dev/dat-schema) project is the latest format. Or try to parse the .dat format from [PoE Dat Viewer](https://snosme.github.io/poe-dat-viewer/) yourself. [RePoE](https://github.com/lvlvllvlvllvlvl/RePoE) collects many common and processed .dat files and exports them to .json format.
3586
36-The problem is, we got to the middle of October, and Path of Exile 2's launch was imminent.
37-And I'll tell you what.
87+### Reference
88+Site | Description
89+- | -
90+[PyPoE](https://github.com/Project-Path-of-Exile-Wiki/PyPoE) | Collection of Python Tools for Path of Exile, mostly used by wiki
91+[RePoE](https://github.com/lvlvllvlvllvlvl/RePoE) | Repository of Path of Exile resources for tool developers.
92+[Poe Dat Viewer](https://snosme.github.io/poe-dat-viewer/) | Web-base Poe Dat Viewer
93+[PoE dat schema](https://github.com/poe-tool-dev/dat-schema) | Source of truth schema for dat files.
94+[Path Of Building Fork schema](https://github.com/PathOfBuildingCommunity/PathOfBuilding/blob/dev/src/Export/spec.lua) | Source of Path Of Building Fork schema
3895
39-Taking people off Path of Exile 2 at that point was just impossible.
40-
41-The endgame was not in a shippable state.
42-
43-But you know what?
44-
45-We could still do it if the Path of Exile 1 team just kept helping until Path of Exile 2 shipped in late November, we could still do it.
46-
47-Maybe the scope would need to decrease a little bit, but mid-February was still possible.
48-
49-We just needed to ship this damn game first.
50-
51-Get it out of the way.
52-
53-We literally couldn't think about anything else until it was done, but we still had time.
54-
55-The problem is, we were fooling ourselves.
56-
57-Path of Exile 2 was delayed three weeks, making the problem even worse.
58-
59-And after the game was released, well, of course we had to immediately deal with all the problems.
60-
61-How could we think about making a Path of Exile 1 expansion?
62-
63-We still have hundreds of thousands of people who are having bad experiences in Path of Exile 2.
64-There are crashes. There are severe endgame balance problems.
65-
66-How could I justify taking some of the most experienced developers we have off Path of Exile 2 when it's on fire?
67-
68-Now, after the Christmas break, we had to really admit something to ourselves.
69-
70-The most urgent priority was getting Path of Exile 2 to a point where it didn't have major issues.
71-
72-And it's hard to even think about what Path of Exile 1 needs when Path of Exile 2 has really obvious problems that need fixing.
73-
74-We needed to get it to the point of stability and balance, and to fix the most severe deficiencies,
75-
76-That Path of Exile 2 has before we can take our eyes off it. So what does that mean?
77-
78-Well, it means that we can't really work on Path of Exile 1 3.26 until Path of Exile 2 0.2.0 has shipped.
79-
80-And then, being honest, we probably need to support that for a couple of weeks after that as well.
81-
82-Now, in the meantime, we are going to get a bunch of pre-production done on 3.26 with whatever people who are not needed to fix issues with Path of Exile 2, and once 0.2.0 is sorted, I promise we will devote as many resources as are needed from our company to get 3.26 done as soon as possible.
83-
84-Unfortunately, I just can't promise a date right now.
85-
86-I'm really sorry for how this went down.
87-
88-Honestly, I should have predicted the fact that taking the Path of Exile 1 team off Path of Exile 1 would lead to this outcome, and I probably should have resisted doing it.
89-
90-But at every stage I just kept thinking, we still have time, we still have time.
91-
92-And then we didn't have time anymore.
93-
94-Now we still have a lot to learn about how to run two games simultaneously.
95-
96-We were overconfident.
97-
98-We still need to work out how to structure our studio to make this possible, but we do have a solid plan for what we want to do with 3.26.
99-
100-Obviously I can't talk about it yet, but I'm hoping you guys are going to love it.
101-
102-I really want to thank you guys for all your support over the years.
103-
104-You guys do deserve better than this and we will get 3.26 out as fast as we can.
96+### PyPoe stable.py to spec.json
97+```
98+from PyPoE.poe.file.specification.data import stable
99+from json import dump
100+with open('D:/spec.json', 'w') as f:
101+ dump(stable.specification.as_dict(), f)
102+```