Use docs from parquet-format as source of truth#142
Use docs from parquet-format as source of truth#142emkornfield merged 8 commits intoapache:productionfrom
Conversation
|
This looks like it might have broken Variant type links, I'll see if I can fix. |
|
Variant types should now be fixed. |
alamb
left a comment
alamb
left a comment
There was a problem hiding this comment.
Thank you @emkornfield -- this is a great step forward
I found only one broken image -- otherwise this is a great step forward
| linkTitle: "Compression" | ||
| weight: 1 | ||
| --- | ||
| ## Overview |
There was a problem hiding this comment.
I rendered this locally and it looks good to me
| @@ -0,0 +1,7 @@ | |||
| --- | |||
There was a problem hiding this comment.
This is really nice to have variant on the webpage now
|
|
||
|
|
||
| [module] | ||
|
|
There was a problem hiding this comment.
nit is that the changes in this file seem unecessary
| @@ -0,0 +1,15 @@ | |||
| <!DOCTYPE html> | |||
There was a problem hiding this comment.
This is pretty clever. It might be worth some comments explaining what it does
There was a problem hiding this comment.
added a comment.
| cost for reading them if it is not doing selective scans. The index structures' | ||
| location and length are stored in ColumnChunk. | ||
|
|
||
|  |
There was a problem hiding this comment.
The image from https://github.com/apache/parquet-format/blob/master/PageIndex.md#technical-approach doesn't seem to be visible anymore:
The image appears to be there in public/images/PageIndexLayout.png but the rendered link is doc/images/PageIndexLayout.png
There was a problem hiding this comment.
Does it mean that all embedded images from apache/parquet-format will be invisible now?
There was a problem hiding this comment.
Yeah, nice catch, let me see what I can do to fix these things up automatically.
There was a problem hiding this comment.
OK, I think this is fixed (it at least seem to be working when I render locally).
| cost for reading them if it is not doing selective scans. The index structures' | ||
| location and length are stored in ColumnChunk. | ||
|
|
||
|  |
There was a problem hiding this comment.
Does it mean that all embedded images from apache/parquet-format will be invisible now?
|
(for anyone following along, relevant mailing list note is here: https://lists.apache.org/thread/qp0ob5z4lvthk94w08d0k9k02ql52fzs) |
|
I just made a PR to update the docs to reflect the latest parquet-format changes. TLDR is that this new system works great 👌 👨🍳 -- thanks again @emkornfield |
3 participants
Before there was duplication of some markdown docs between
parquet-siteandparquet-format, this adds parquet-format as a submodule and an rule that can import from the submodule.Note, staging seems quite out of date but if desired we can try this out there first (I tested locally with docker and this seems to produce reasonable results).