Question / Comment: bbox doesn't exactly match the word

[victor-ab]

For some documents (just a few), the bbox doesn't exactly match the word.

doc = fitz.open(pdf_dir)
pages = [ doc[ i ] for i in range( doc.pageCount ) ]
page = pages[0]

blocks = page.getText("dict")['blocks']
text_blocks = [block for block in blocks if block['type']==0]
color='red'
width=2
TARGET_DPI = 300
mat = fitz.Matrix(TARGET_DPI/ 72, TARGET_DPI/ 72)
pix_map = doc.getPagePixmap(0,matrix=mat)
image = Image.open(BytesIO(pix_map.getImageData()))
draw = ImageDraw.Draw(image)

for block in text_blocks:
    for lines in text_blocks:
        for line in lines['lines']:
            for span in line['spans']:
                x0, y0, x1, y1 = span['bbox']
                x0, y0, x1, y1 = x0*TARGET_DPI/ 72, y0*TARGET_DPI/ 72, x1*TARGET_DPI/ 72, y1*TARGET_DPI/ 72
                draw.rectangle((x0, y0, x1, y1), fill=None, outline='green', width=width)

If I add y0 = y1-span['size'], then it works:

Why does it happens? Is it safer to always use the size?

[JorjMcKie]

I guess I don't really understand your code:
Why do you not directly use the span bbox as the coordinates of the rectangle to draw? Instead you enlarge it by 300/72 = 4.17.
If the following doesn't match, we have a real issue and I would need a reproducer:
page.drawRect(span["bbox"], width=0.3, color=green).

Apart from that argument, I have seen font examples, which do not correctly return the character bbox (this is the "atom", the larger bboxes for span, line and block are computed by joining these).
The char bbox is computed by MuPDF - which uses font properties for this. If the font has problems, then the result is an incorrect char bbox of course. In the mentioned example, tha char bbox had a height of zero, So my code now checks this and uses the font size in that case ...
To cut a long story short: if my previous instruction produces incorrect rectangles, please send me the example.

[victor-ab]

I am sorry, this ratio 300/72 is just for enlarging the dpi and matching it with the bounding box coordinates.
Had the same output with drawRect.
raw-Copy.pdf

I edited the pdf with Wondershare PDFelement, but the problem remains:

[JorjMcKie]

Indeed font issues. I made an adjustment which ensures that span bboxes can not have a smaller height than the fontsize.
This snippet leads to the following picture:

>>> for block in page.getText("dict",flags=0)["blocks"]:
	for line in block["lines"]:
		for span in line["spans"]:
			_ = page.drawRect(span["bbox"], width=0.3, color=(1,0,0))

An irritation remains:
Your example PDF has two fonts - well-known ones, at least per name. And both show the same problem ...
The following things also require more consideration:

• I applied my fix only to span heights (which percolates up to line an block heights)
• single character bboxes still have a too small height, because I do not want all of them as high as an uppercase character.
• Other extraction types, speficially getText("words"), remain to be treated accordingly.

[victor-ab]

@JorjMcKie What was the applied change?
Just adding that flag 0? If so, it did not worked here.

[JorjMcKie]

No it was just "your" fix: making sure the span bbox height is not less than fontsize - however in the C-module behind gettext("dict").
The flags=0 just make sure that no images are extracted, that ligatures ("fi", "ffi", etc.) are decomposed and that white space is treated as blanks.
I am now developing an algorithm which generalizes the height correction on a by-character level - which is only trivial for horizontal writing mode and zero inclination wrt the x-axis.

[JorjMcKie]

Finished a stable version, which resorts to "your fix" if the character bbox height is smaller than the fontsize.
This version should work correctly if text is in horizontal write mode and rotated by some int multiple of 90 degrees.

This should cover the vast majority of all cases. Also dealing with non-orthogonal text inclinations is far more complicated, allthough certainly possible.
To give you an impression that the problem is the font (not PyMuPDF / MuPDF) compare the outputs of your page:

• Number 1 with the repaired bbox height. Note how narrow the boxes around text are ...
  

• Number 2 after I replaced the font by Helvetica. The boxes are a lot more "generous" now!
  

[victor-ab]

Glad to hear you are making progress!

That's also a very interesting aspect about bounding boxes.

Why aren't the bboxes always narrow around the text? Even in a PDF with "good fonts", They always seems to be kinda "off".
Is there an option to control the margin of the bounding box?

There's even an overlap sometimes

[JorjMcKie]

MuPDF aims to return the /Lineheight of the text. It usually (not always) is a number in the range of fontsize * 1.2 or 1.3. Sometimes larger.
This information piece equals (for well-defined fonts) the height of the so-called font bbox (in addition, every glyph may have its own bbox).
The lineheight includes the /Descent and the /Ascent plus some value for the distance to the preceeding line. The descender is a negative number indicating the amount of pixels some characters go below the baseline, e.g. "y" or "j" or "g". The ascender is the height of the highest glyph above the baseline - including those super-high characters like |.
In PDF these numbers are given as integers and must be divided by 1000 and then multiplied with fontsize to give th actual values.
The fontsize is counted from the character descender upwards to include capital letters.

An additional information bit is the origin, a point contained in PyMuPDF's "dict" output per each character (and on the span level we have the lowest such value across all characters of the span).
The origin indicates the lower-left corner of each character. It is not equal to the lower-left bbox corner, of neither the span nor of the character. It should be the y-value of all characters' baseline.
If you get a hold of the font's ascender and descender values, e.g. /Ascent = 953, /Descent = -285, do this calculation to get a modified span bbox with minimal height:

a = ascent / 1000 * span["size"]
d = descent / 1000 * span["size"]
o = fitz.Point(span["origin"])
r = fitz.Rect(span["bbox"])
r.y1 = o.y - span["size"] * d / (a - d)  # d is negative!
r.y0 = r.y1 - span["size"]

Note:
If you have access to the fontfile or fontbuffer, or can otherwise construct the correct font (e.g. by font = fitz.Font("helv")), you have access to those two values via properties font.ascender and font.descender, and the above calculation are simply a = font.ascender * span["size"].

[JorjMcKie]

with the algorithm pinned down in previous post, we get the following "optimized" span bboxes for the well-behaving Helvetica font:

[JorjMcKie]

Watch how "Currency" bbox includes the descending character "y"!
The other bboxes still seem a bit large if they have no descending character. Being awkward, you could resort to "rawdict" and do your own calculation of each character's bbox (fingers crossed that the font is super-wellbehaving and provides individually different such bboxes!) ...

[victor-ab]

Hmmm, interesting.

How can I exactly identify which font from the document is being used in a specific span? The fontName extracted from rawdict is kinda generic.
For instance, in another pdf, the fontName in a span is just Arial. But when I extract the font list, I get:

>>> page.getFontList()
[(6, 'ttf', 'Type0', 'IHBWDB+Arial', 'F1', 'Identity-H'),
 (18, 'ttf', 'Type0', 'IHBWDB+Arial', 'F2', 'Identity-H'),
 (26, 'ttf', 'Type0', 'IHBWDB+ArialNarrow', 'F3', 'Identity-H')]

But, unfortunately, it is very hard to believe that most of fonts will be "super-wellbehaving".
Here is an example:

for block in text_blocks:
    for lines in text_blocks:
        for line in lines['lines']:
            for span in line['spans']:
                for char in span['chars']:
                    page.drawRect(char['bbox'], width=0.3, color=(0,1,1))

In this case, even the y0 is off.

[JorjMcKie]

Font names starting with 6 upper case letter like IHBWDB+Arial are subsets of the same font, the name of which follows the "+".
The two fonts with the same name, but under different xrefs, 6, 18, must be the same by PDF spec!
They appear in different xrefs, because that PDF is badly optimized.
The buffers stored for xref 6, resp. 18, MUST be the same, or someone made a major error.
To get a clearer picture, you may want to do a page.cleanContents() before making a fontlist or extract text.

I once did include that 6-letter prefix in getText() but dropped it again at some point, b/o no value added information.
And you should receive "ArialNarrow" if that font is indeed used on the page - which need not be the case just because that font appears in the list!
So, what you could do is searching the fontlist for a name ending with the getText() fontname - sometimes a more sophisticated search is required, because you are actually looking for the name of the basefont ...
Then extract the fontbuffer via the xref and make a temporary font to find out those ascender/descender properties.

Well, and if all fails, and the font contains no character-specific bbox, you will have to live with the font level one. The info simply does not exist then.

In this case, even the y0 is off.

What do you mean, the picture looks ok to me.

[victor-ab]

Got it, thanks a lot for the comprehensive explanation!
I guess it will be better to just live with that. If two bboxes overlaps, I'll just change move overlapping boundary to the middle of that overlap.

In this case, even the y0 is off.

I meant that the y0 is higher than the upper-case character.