documentation: update PDF design doc

site/dev/design/pdftheory.md

 PDF Theory of Operation
 =======================
 
 <!--
 PRE-GIT DOCUMENT VERSION HISTORY
     2012-06-25 Steve VanDeBogart
                * Original version
     2015-01-14 Hal Canary.
                * Add section "Using the PDF backend"
                * Markdown formatting
@@ skipping 12 matching lines @@
 
 ### Contents ###
 
 *   [Typical usage of the PDF backend](#Typical_usage_of_the_PDF_backend)
 *   [PDF Objects and Document Structure](#PDF_Objects_and_Document_Structure)
 *   [PDF drawing](#PDF_drawing)
 *   [Interned objects](#Interned_objects)
 *   [Graphic States](#Graphic_States)
 *   [Clip and Transform](#Clip_and_Transform)
 *   [Generating a content stream](#Generating_a_content_stream)
-*   [Margins and content area](#Margins_and_content_area)
 *   [Drawing details](#Drawing_details)
     +   [Layers](#Layers)
     +   [Fonts](#Fonts)
     +   [Shaders](#Shaders)
     +   [Xfer modes](#Xfer_modes)
 *   [Known issues](#Known_issues)
 
 <a name="Typical_usage_of_the_PDF_backend"></a>
 Typical usage of the PDF backend
 --------------------------------
 
 SkPDFDevice is the main interface to the PDF backend. This child of
-SkDevice can be set on an SkCanvas and drawn to. It requires no more
-care and feeding than SkDevice. Once drawing is complete, the device
-should be added to an SkPDFDocument as a page of the desired PDF. A
-new SkPDFDevice should be created for each page desired in the
-document. After all the pages have been added to the document,
-`SkPDFDocument::emitPDF()` can be called to get a PDF file. One of the
-special features of the PDF backend is that the same device can be
-added to multiple documents. This for example, would let you generate
-a PDF with the single page you just drew as well as adding it to a
-longer document with a bunch of other pages.
-
-<!--?prettify lang=cc?-->
-
-    SkPDFCanon canon;
-    SkAutoUnref<SkPDFDevice> pdfDevice(
-        SkPDFDevice::Create(SkISize::Make(width, height), 72.0f, &canon));
-
-    SkCanvas canvas(pdfDevice);
-    draw_content(&canvas);
-
-    SkPDFDocument doc;
-    doc.appendPage(dev);
-    doc.emitPDF(&pdf_stream);
+SkDevice can be set on an SkPDFCanvas and drawn to.  Once drawing is
+complete, the device's content and resouces are added to the
+SkPDFDocument that owns the device.  A new SkPDFDevice should be
+created for each page or layer desired in the document.  After all the
+pages have been added to the document, `SkPDFDocument::onClose()` is
+called to finish the PDF file.
 
 <a name="PDF_Objects_and_Document_Structure"></a>
 PDF Objects and Document Structure
 ----------------------------------
 
 ![PDF Logical Document Structure](/dev/design/PdfLogicalDocumentStructure.png)
 
 **Background**: The PDF file format has a header, a set of objects and
 then a footer that contains a table of contents for all of the objects
 in the document (the cross-reference table). The table of contents
 lists the specific byte position for each object. The objects may have
 references to other objects and the ASCII size of those references is
 dependent on the object number assigned to the referenced object;
 therefore we can’t calculate the table of contents until the size of
-objects is known, which requires assignment of object
-numbers.
+objects is known, which requires assignment of object numbers.  The
+document uses SkWStream::bytesWritten() to query the offsets of each
+object and build the cross-reference table.
 
 Furthermore, PDF files can support a *linearized* mode, where objects
 are in a specific order so that pdf-viewers can more easily retrieve
 just the objects they need to display a specific page, i.e. by
 byte-range requests over the web. Linearization also requires that all
 objects used or referenced on the first page of the PDF have object
 numbers before the rest of the objects. Consequently, before
 generating a linearized PDF, all objects, their sizes, and object
 references must be known.  Skia has no plans to implement linearized
 PDFs.
 
-<!-- <del>At this point, linearized PDFs are not generated. The
-framework to generate them is in place, but the final bits of code
-have not been written.</del> -->
-
     %PDF-1.4
     …objects...
     xref
     0 31  % Total number of entries in the table of contents.
     0000000000 65535 f
     0000210343 00000 n
     …
     0000117055 00000 n
     trailer
     <</Size 31 /Root 1 0 R>>
     startxref
     210399  % Byte offset to the start of the table of contents.
     %%EOF
 
-The class SkPDFCatalog and the virtual class SkPDFObject are used to
+The class SkPDFObjNumMap and the virtual class SkPDFObject are used to
 manage the needs of the file format. Any object that will represent a
 PDF object must inherit from SkPDFObject and implement the methods to
 generate the binary representation and report any other SkPDFObjects
 used as resources. SkPDFTypes.h defines most of the basic PDF objects

[tomhudson, 2016/09/01 19:11:58]

Nit: object shouldn't be plural here.

[hal.canary, 2016/09/28 18:18:28]

Done.

-types: bool, int, scalar, string, name, array, dictionary, and object
-reference. The stream type is defined in SkPDFStream.h. A stream is a
+types: bool, int, scalar, string, name, array, dictionary, and stream.
+A stream is a
 dictionary containing at least a Length entry followed by the data of
 the stream. All of these types except the stream type can be used in
 both a direct and an indirect fashion, i.e. an array can have an int

[tomhudson, 2016/09/01 19:11:58]

Nit: this isn't clear at all. Most of the rest of this paragraph isn't, either.

[hal.canary, 2016/09/28 18:18:28]

Acknowledged.

 or a dictionary as an inline entry, which does not require an object
 number. The stream type, cannot be inlined and must be referred to
 with an object reference. Most of the time, other objects types can be
 referred to with an object reference, but there are specific rules in
 the PDF specification that requires an inline reference in some place
 or an indirect reference in other places. All indirect objects must
 have an object number assigned.
 
 *   **bools**: `true` `false`
 *   **ints**: `42` `0` `-1`
 *   **scalars**: `0.001`
 *   **strings**: `(strings are in parentheses or byte encoded)` `<74657374>`
 *   **name**: `/Name` `/Name#20with#20spaces`
 *   **array**: `[/Foo 42 (arrays can contain multiple types)]`
 *   **dictionary**: `<</Key1 (value1) /key2 42>>`
 *   **indirect object**:  
     `5 0 obj  
     (An indirect string. Indirect objects have an object number and a
     generation number, Skia always uses generation 0 objects)  
     endobj`  
 *   **object reference**: `5 0 R`
 *   **stream**: `<</Length 56>>  
     stream  
     ...stream contents can be arbitrary, including binary...  
     endstream`
 
 The PDF backend requires all indirect objects used in a PDF to be
-added to the SkPDFCatalog of the SkPDFDocument. The catalog is
+added to the SkPDFObjNumMap of the SkPDFDocument. The catalog is
 responsible for assigning object numbers and generating the table of
 contents required at the end of PDF files. In some sense, generating a
 PDF is a three step process. In the first step all the objects and
 references among them are created (mostly done by SkPDFDevice). In the
-second step, object numbers are assigned and SkPDFCatalog is informed
-of the file offset of each indirect object. Finally, in the third
+second step, SkPDFObjNumMap assigns and remembers object numbers.
+Finally, in the third
 step, the header is printed, each object is printed, and then the
 table of contents and trailer are printed. SkPDFDocument takes care of
 collecting all the objects from the various SkPDFDevice instances,
-adding them to an SkPDFCatalog, iterating through the objects once to
+adding them to an SkPDFObjNumMap, iterating through the objects once to
 set their file positions, and iterating again to generate the final
 PDF.
 
+As an optimization, many leaf nodes in the direct graph of indirect
+objects can be assigned object numbers and serialized early.
+
     %PDF-1.4
     2 0 obj <<
       /Type /Catalog
       /Pages 1 0 R
     >>
     endobj
     3 0 obj <<
       /Type /Page
       /Parent 1 0 R
       /Resources <>
@@ skipping 72 matching lines @@
 Interned objects
 ----------------
 
 There are a number of high level PDF objects (like fonts, graphic
 states, etc) that are likely to be referenced multiple times in a
 single PDF. To ensure that there is only one copy of each object
 instance these objects an implemented with an
 [interning pattern](http://en.wikipedia.org/wiki/String_interning).
 As such, the classes representing these objects (like
 SkPDFGraphicState) have private constructors and static methods to
-retrieve an instance of the class. Internally, the class has a list of
-unique instances that it consults before returning a new instance of
-the class. If the requested instance already exists, the existing one
-is returned. For obvious reasons, the returned instance should not be
-modified. A mechanism to ensure that interned classes are immutable is
-needed.  See [issue 2683](https://bug.skia.org/2683).
+retrieve an instance of the class.
+
+The SkPDFCanon object owns the interned objects.  For obvious reasons,
+the returned instance should not be modified. A mechanism to ensure
+that interned classes are immutable is needed.  See [issue
+2683](https://bug.skia.org/2683).
 
 <a name="Graphic_States"></a>
 Graphic States
 --------------
 
 PDF has a number of parameters that affect how things are drawn. The
 ones that correspond to drawing options in Skia are: color, alpha,
 line cap, line join type, line width, miter limit, and xfer/blend mode
 (see later section for xfer modes). With the exception of color, these
 can all be specified in a single pdf object, represented by the
@@ skipping 52 matching lines @@
 SkPDFGraphicState (interned) with the rest (color, font size, etc)
 explicitly stored in the ContentEntry. After populating the
 ContentEntry with all the relevant context, it is compared to the the
 most recently used ContentEntry. If the context matches, then the
 previous one is appended to instead of using the new one. In either
 case, with the context populated into the ContentEntry, the
 appropriate draw call is allowed to append to the content stream
 snippet in the ContentEntry to affect the core of the drawing call,
 i.e. drawing a shape, an image, text, etc.
 
-When all drawing is complete, SkPDFDocument::emitPDF() will call
+When all drawing is complete, SkPDFDocument::onEndPage() will call
 SkPDFDevice::content() to request the complete content stream for the
 page. The first thing done is to apply the initial transform specified
 in part in the constructor, this transform takes care of changing the
 coordinate space from an origin in the lower left (PDF default) to the
 upper left (Skia default) as well as any translation or scaling
 requested by the user (i.e. to achieve a margin or scale the
 canvas). Next (well almost next, see the next section), a clip is
 applied to restrict drawing to the content area (the part of the page
 inside the margins) of the page. Then, each ContentEntry is applied to
 the content stream with the help of a helper class, GraphicStackState,
 which tracks the state of the PDF graphics stack and optimizes the
 output. For each ContentEntry, commands are emitted to the final
 content entry to update the clip from its current state to the state
 specified in the ContentEntry, similarly the Matrix and drawing state
 (color, line joins, etc) are updated, then the content entry fragment
 (the actual drawing operation) is appended.
 
-<a name="Margins_and_content_area"></a>
-Margins and content area
-------------------------
-
-The above procedure does not permit drawing in the margins. This is
-done in order to contain any rendering problems in WebKit. In order to
-support headers and footers, which are drawn in the margin, a second
-set of ContentEntry’s are maintained. The
-methodSkPDFDevice::setDrawingArea() selects which set of
-ContentEntry’s are drawn into. Then, in the SkPDFDevice::content()
-method, just before the clip to the content area is applied the margin
-ContentEntry's are played back.
-
-<!-- TODO(halcanary): update this documentation. -->
-
 <a name="Drawing_details"></a>
 Drawing details
 ---------------
 
 Certain objects have specific properties that need to be dealt
 with. Images, layers (see below), and fonts assume the standard PDF
 coordinate system, so we have to undo any flip to the Skia coordinate
 system before drawing these entities. We don’t currently support
 inverted paths, so filling an inverted path will give the wrong result
 ([issue 241](https://bug.skia.org/241)). PDF doesn’t draw zero length
 lines that have butt of square caps, so that is emulated.
 
 <a name="Layers"></a>
 ### Layers ###
 
 PDF has a higher level object called a form x-object (form external
 object) that is basically a PDF page, with resources and a content
 stream, but can be transformed and drawn on an existing page. This is
-used to implement layers. SkDevice has a method,
-createFormXObjectFromDevice, which uses the SkPDFDevice::content()
+used to implement layers. SkPDFDevice has a method,
+makeFormXObjectFromDevice(), which uses the SkPDFDevice::content()
 method to construct a form x-object from the the
 device. SkPDFDevice::drawDevice() works by creating a form x-object of
 the passed device and then drawing that form x-object in the root
 device. There are a couple things to be aware of in this process. As
 noted previously, we have to be aware of any flip to the coordinate
 system - flipping it an even number of times will lead to the wrong
 result unless it is corrected for. The SkClipStack passed to drawing
 commands includes the entire clip stack, including the clipping
 operations done on the base layer. Since the form x-object will be
 drawn as a single operation onto the base layer, we can assume that
@@ skipping 34 matching lines @@
 PDF only supports an 8-bit encoding for Type1 or Type3 fonts. However,
 they can contain more than 256 glyphs. The PDF backend handles this by
 segmenting the glyphs into groups of 255 (glyph id 0 is always the
 unknown glyph) and presenting the font as multiple fonts, each with up
 to 255 glyphs.
 
 #### *Font subsetting* ####
 
 Many fonts, especially fonts with CJK support are fairly large, so it
 is desirable to subset them. Chrome uses the SFNTLY package to provide
-subsetting support to Skia for TrueType fonts. However, there is a
-conflict between font subsetting and interned objects. If the object
-is immutable, how can it be subsetted? This conflict is resolved by
-using a substitution mechanism in SkPDFCatalog. Font objects are still
-interned, but the interned objects aren’t internally
-populated. Subsetting starts while drawing text to an SkPDFDevice; a
-bit set indicating which glyphs have been used is maintained. Later,
-when SkPDFDocument::emitPDF() is rendering the PDF, it queries each
-device (each page) for the set of fonts used and the glyphs used from
-each font and combines the information. It then asks the interned
-(unpopulated) font objects to create a populated instance with the
-calculated subset of the font - this instance is not interned. The
-subsetted instance is then set as a substitute for the interned font
-object in the SkPDFCatalog. All future references to those fonts
-within that document will refer to the subsetted instances, resulting
-in a final PDF with exactly one instance of each used font that
-includes only the glyphs used.
-
-The substitution mechanism is a little complicated, but is needed to
-support the use case of an SkPDFDevice being added to multiple
-documents. If fonts were subsetted in-situ, concurrent PDF generation
-would have to be explicitly handled. Instead, by giving each document
-its own subsetted instance, there is no need to worry about concurrent
-PDF generation. The substitution method is also used to support
-optional stream compression. A stream can used by different documents
-in both a compressed and uncompressed form, leading to the same
-potential difficulties faced by the concurrent font use case.
+subsetting support to Skia for TrueType fonts.
 
 <a name="Shaders"></a>
 ### Shaders ###
 
 Skia has two types of predefined shaders, image shaders and gradient
 shaders. In both cases, shaders are effectively positioned absolutely,
 so the initial position and bounds of where they are visible is part
 of the immutable state of the shader object. Each of the Skia’s tile
 modes needs to be considered and handled explicitly. The image shader
 we generate will be tiled, so tiling is handled by default. To support
@@ skipping 89 matching lines @@
 Dst (or not). SrcIn is SrcMode with Src drawn with Dst as a
 mask. SrcOut is like SrcMode, but with Src drawn with an inverted Dst
 as a mask. DstIn is SrcMode with Dst drawn with Src as a
 mask. Finally, DstOut is SrcMode with Dst draw with an inverted Src as
 a mask.
 
 <a name="Known_issues"></a>
 Known issues
 ------------
 
-*   [issue 241](https://bug.skia.org/241)
-    As previously noted, a boolean geometry library
-    would improve clip fidelity in some places, add supported for
-    inverted fill types, as well as simplify code.
-    This is fixed, but behind a flag until path ops is production ready.
 *   [issue 237](https://bug.skia.org/237)
     SkMaskFilter is not supported.
 *   [issue 238](https://bug.skia.org/238)
     SkColorFilter is not supported.
 *   [issue 249](https://bug.skia.org/249)
     SrcAtop Xor, and Plus xfer modes are not supported.
 *   [issue 240](https://bug.skia.org/240)
     drawVerticies is not implemented.
 *   [issue 244](https://bug.skia.org/244)
     Mostly, only TTF fonts are directly supported.  (User metrics 

[tomhudson, 2016/09/01 19:11:58]

Nit: mostly only?

[hal.canary, 2016/09/28 18:18:28]

Acknowledged.

-    show that almost all fonts are truetype.
+    show that almost all fonts are truetype.)
 *   [issue 260](https://bug.skia.org/260)
     Page rotation is accomplished by specifying a different
     size page instead of including the appropriate rotation
     annotation.
 
 * * *