apply_redactions causes part of the page content to be hidden / transparent

[beeing]

Description of the bug

I'm adding a redaction region to a part of the PDF, but after calling apply_redactions(), one side of the entire page goes missing (opened in macOS Preview app or Safari).

Further inspection reveals that the text is not missing, as it is selectable and can be copied out properly. It is either the text has been masked / hidden, but I could not find out how to check further (sorry, my limited knowledge on PDF structure).

The media, crop, art, bleed, trim boxes all looks fine before and after the redactions. In fact, I'm trying to check if there's other paths, objects that may be causing it but there's nothing.

Note that I'm not able to share the actual PDF but it was generated from Puppeteer / Chromium (PDF ver 1.7).

Thanks in advance for looking into this.

How to reproduce the bug

1. Generate PDF from Chromium / Puppeteer
2. Add redaction of any size eg. (0,0,1,1) and call apply_redactions()
3. Open the PDF in Preview App.

PyMuPDF version

1.24.9

Operating system

MacOS

Python version

3.9

[Kyrylo-Hrytsenko]

I faced the same problem.
Note: this problem does not exist on version 1.23.9, all higher versions have it

There is also an interesting thing. If you open redacted PDF via Chrome or LibreOffice it will look as expected. But the issue is reproducible at least with Mac Preview and react-pdf-viewer lib.

[JorjMcKie]

As always: please provide an example PDF! There is no way to otherwise deal with this post.

[beeing]

I've just tested and it works on older version (up to pymupdf-1.23.26).

Perhaps easier to compare the commits since a868c0a until HEAD.

[Kyrylo-Hrytsenko]

@JorjMcKie
Here is an example
original.pdf
redacted.pdf

Please, open the redacted file with the Preview app, it will look like this

The code for redaction looks like this:

pdfIn = fitz.open(input_file)

out_buffer = BytesIO()

page = pdfIn[0]

page.add_redact_annot([0,0,100,100], text=None, fill=(0, 0, 0))
page.add_redact_annot([100,100,200,200], text=None, fill=(0, 0, 0))
page.add_redact_annot([200,200,300,300], text=None, fill=(0, 0, 0))
page.add_redact_annot([300,300,400,400], text=None, fill=(0, 0, 0))

page.apply_redactions()

pdfIn.save(out_buffer, garbage=3, deflate=True)
pdfIn.close()

with open(output_file, mode='wb') as f:
    f.write(out_buffer.getbuffer())
f.close()

[Kyrylo-Hrytsenko]

@JorjMcKie
May I ask if you have received the PDF for reproducing the issue?

[JorjMcKie]

@Kyrylo-Hrytsenko Thanks, I did.

I executed the script and found no problem at all using v1.24.9. I modified the script somewhat so redaction rectangles are visible and erased areas are not filled:

import pymupdf

print(pymupdf.version)
pdfIn = pymupdf.open("original.pdf")

page = pdfIn[0]
rects = (
    [0, 0, 100, 100],
    [100, 100, 200, 200],
    [200, 200, 300, 300],
    [300, 300, 400, 400],
)
for r in rects:
    page.draw_rect(r, color=(1, 0, 0))
    page.add_redact_annot(r)

page.apply_redactions()

pdfIn.ez_save("output.pdf")

Gives this correct result:
output.pdf

[Kyrylo-Hrytsenko]

@JorjMcKie
Your result file looks like this for me in the Preview app:

Notes:

• I highlighted some text with the mouse to show that it is still present in the PDF, but for some reason, it's hidden.
• There is no issue when I open this file in Chrome or any other app.
• I checked version 1.23.9, and there was no such issue. It seems to have started happening in version 1.24.0.

Does the output.pdf look normal when you open it in the 'Preview' application?

[JorjMcKie]

I do not use or have Preview.
My file is displayed in all PDF viewers like Adobe Acrobat, Foxit, Nitro, PDF XChange, evince (Linux). So all authoritative applications behave correctly.
No idea what is wrong with Preview.

[jamie-lemon]

Can confirm also see this problem in Preview. However it is fine when I open in Adobe Acrobat. To me this feels like a Preview rendering bug. I would submit a bug to Apple if that is possible!