Work with PDF files

Some PDF files are just a collection of images.

This is particularly true for scans, and comic books.

In those cases, I prefer a simpler format like .cbz, which is just a renamed .zip archive containing images.

In my experience, .cbz files are easier to modify (just extract the images and work on them directly), and on ebook readers, the program rendering the .cbz files does normally a better job at cropping blank borders.

For extracting all images, see the following commands

Unless you have a particular requirement, either use -all or a specific format (use pdfimages --help to see the list of supported formats).

If no format is specified, pdfimages will convert all images to .ppm files.

I wondered long enough why a 33.5MiB PDF produces more than 300MiB of images, which resulted in 175MiB compressed .png images.

For what is worth, in one case, extracting (no conversions) images from a 33.5MB PDF file resulted in 33.6MiB images. As the PDF format does not solely consist of images, but some additional structure and metadata, it seemed strange that the images alone did occupy slightly more space.

If you look closely at the output of pdfimages -list, the last column will report if an image is compressed losslessly inside the PDF structure, and in my case, all images were compressed:

Creating a zip archive with all the images (with some default settings for the compression level) resulted in 32.2MiB, and stripping the metadata I was not interested in resulted in 31.6MiB, compressed in a zip archive to 31.1MiB.

It is not a great achievement in terms of size (just reduce it by 4%), but I have a format that is easier to deal with, with no loss in quality.

You should ensure that your comics actually consist only of images. Some .pdf comics consist of two parts; the images (as images), and the text as text. The effect is that the extracted images would have, for example, empty balloon dialogs.

On some documents, a page is composed of multiple images, thus after extracting them one should, somehow, convert them to a single image.

Another thing to pay attention to is that even if the whole page consists of an image, it might contain useful metadata. Some scanned documents have an invisible overlay (eventually created with some OCR program), that makes it possible to copy the scanned text. It should not be necessary to say that extracting the images will not preserve the copyable text.

Thus before converting a PDF file to .cbz, ensure that you are not interested in the additional metadata.

Some comics do not have one image per page, but multiple composed and overlayed together. Or as mentioned earlier, the text could be not part of the image itself.

In that case, extracting all images will not yield the desired result.

Even if it is a lossy conversion, one might still prefer to have images instead of a PDF file, in that case, it is possible to convert every page to an image file

In this case, the difference in size can vary much more compared to pdfimages -all, as the conversion can alter the quality of the content.

diffpdf is, as far as I know, the most reliable program for comparing visual differences in PDF files.

It is relatively easy to use and can show if there are any visual differences (images included), or if only the text (words or single characters) differ (eventually ignoring fonts).

It will highlight where those differences are, either by marking them or by showing the difference between one file and another, instead of the two files side by side (which is very practical for documents with images).

Comparing two PDF files is important if you want to ensure that some operation did not modify its content.

I’ve used, for example, diffpdf as part of a pipeline to ensure that the output of generated documents was unchanged after optimizing the build process and cleaning up the source files.

In that case, I did not care if some part of the appearance was different, but mainly if the words or letters did change.

I currently have no good solution for those use cases.

The most common scenario is having a document to sign and send electronically.

The "official" way, excluding a digital signature which is something else, would be to print the document, sign the printed page, and then scan it.

The more environmentally friendly solution is to sign a piece of paper, scan it, and save it as an image somewhere on your PC. Every time you need to sign a document, instead of printing it, insert the image at the appropriate position.

LibreOffice Draw can accomplish this task, but converting between a PDF file and its internal format, will generally (slightly) change your document.

Some elements might not be aligned as before anymore, or the used might be different.

If you are not sure, use diffpdf. Ideally, the file should look identical except where I placed the signature or additional text.

Most of the time the differences are minor, especially if you adjust alignment issues by hand, and unless one is looking at two documents side by side, the difference will be hard to notice

As of today, no one asked me why the returned documents look slightly different, I assume that no one noticed, or at least that no one cared enough to make an issue out of it.

Nonetheless, there should be a better way.

A way that

I’ve yet not found a program that ticks all the boxes, suggestions are welcome.

I’ve recently tried xournal++; it seems to do a better job than LibreOffice Draw but depending on the document it also introduces some unwanted changes.

If you have two files, and you want to append one to another, then there are multiple tools at your disposal, especially from the command line.

TODO: not all suggested programs are equivalent. In particular, some might preserve metadata from the original document, and do different things in front of pages with different layouts and internal links.

According to pdfinfo:

Size of input file: 2839681 bytes

How can qpdf create such a small file? It deduplicates all objects. Since in this test, I appended a file to itself, most internal objects can be reused.

I did otherwise not validate output.pdf file, nor use a particularly complex document.

TODO: not all suggested programs are equivalent. In particular, some might preserve metadata from the original document, and do different things in front of pages with different layouts and internal links.

According to pdfinfo:

I did not validate output.pdf, or used a particularly complex document during my tests.

The output will look similar to

Note that a document has generally much more metadata than those shown with pdfinfo and other tools.

Some documents are huge.

PDF files can have complex structures, and there are many ways to represent certain content.

In addition to that, elements inside a PDF file can be compressed.

Thus it should be possible to "optimize" the size of a document, just like jpegoptim and optipng do for .jpg and .png files.

The first one is to remove all elements that are not visible, and eventually metadata too.

The second method would be to represent more efficiently the content.

As of today, all methods tried (except one) failed. By "failed" I mean that the optimization was not lossless, although I might not have been able to detect it with the naked eye.

Again, diffpdf will help to validate if a conversion was lossless or not.

In general, gs and ps2pdf are lossy, and they can help if the document has hidden content (metadata too) or images that are at a too high resolution.

I’m not sure why exiftool is presented as a tool capable of removing metadata, it is not. It can add additional empty metadata and thus increases the size.

While it is true that pdfinfo and most other tools only show the last metadata, the original metadata is still there in cleartext.

There are mainly two ways to compress PDF files. With lossy or lossless methods.

In this context, lossless means that the rendered content is identical, and that text remains text (thus it can still be copied). Unused objects and invisible content might be removed, and the internal structure of the PDF document might look completely different.

Lossy means that there might be some artifacts, and text might be converted to an image.

The only method I’ve found for doing a lossless compression is qpdf:

There is also the parameter --optimize-images, which (I suppose) will make a lossy compression of the images.

Not all programs can handle PDF documents with passwords, and I do not want to decrypt those documents every time I want to read them.

So I generally save a decrypted copy

PDF files might have restrictions; for example not being able to print them, or not being able to select and copy the text.

Those restrictions are handled by the PDF reader, thus using a PDF reader that does not restrict what the end-user can do is sufficient.

But if you have to work with a given program that applies those restrictions, you might want to remove them.

The "official" way is to use the "owner password" (not to be confused with the "user password", used for decrypting a password-protected file), but you do not need to know it

Instead of printing one (small) image per page, I find it much more logical to print as many images as possible at once.

Normally I use LibreOffice Writer, but I assume that Microsoft Word is equally good.

In fact, for my main use case, I do not even need to create a PDF document if the device attached to the printer supports .odt, .doc, or .docx documents.

PDF files can act more or less like zip archives, and contain other files.

qpdf has the command-line options --list-attachments and --show-attachment, which can be used for extracting all embedded files.

Even if a PDF reader can open, process, and show the content of a document, it does not mean that the file does not have some invalid structure.

Most programs can handle malformed PDF documents, but it would be better not to have such documents, especially if you are dealing with digital signatures, have created such documents to send to other people, or want to be sure to be able to open such documents in the future.

qpdf has an option for verifying if a PDF document is valid: qpdf --check input.pdf

It will not be able to verify all possible types of error, but it seems to verify multiple properties.

If you want to repair a document, then processing it with any of the mentioned tools (qpdf, pdftk, pdfjam, pdfunite, …​) should repair it, as they generally parse a document and create a new one.

For example, qpdf input.pdf output.pdf will create a valid output.pdf file, even if input.pdf is a partially broken PDF file.

If you want to read a PDF file from the command line, or if you have some automated process that depends on the content of a PDF file, you might want to extract the text from it.

On the documents I’ve tested, pdftotext seems to give better results, as the displayed text has a "better" layout, that also resembles the layout used in the PDF file.

Note that both programs have some options for defining the layout of the output, and for creating HTML files.

There is pdfarranger:

Xournal++ is a note-taking software which is also able import, modify and export PDF files. Contrary to LibreOffice Draw, the resulting document seems to be much more similar to the original, althoguh the resulting document might still have some artifacts.

And tabula-java, which is both a (Java) library and a command line tool specialized in extracting tabular data from PDF files.

And some document readers: