Dewarping / Line Item Extraction / Text Reading Order

[Razlaw]

Hey :)
Thanks again for all the great work on the project.
I'm currently looking into the topic of line item extraction.
This already works well for photos of flat paper taken from a top-down perspective.
Rotations are properly corrected - especially after using your finetuning script (thanks again :) ).
But other perspectives - e.g. from the side - and curved or wrinkled paper are still a problem.

Top-down perspective, lines in orange with their index as blue numbers:

Same photographed receipt, shifted perspective causes wrong line ordering:

I'm well aware that this is a hard task to solve in a general way.
@felixdittrich92 do you have some pointers to repos, papers, anything you see potential in?
I could try integrate it into the orientation estimation functionality.

All others are welcome to comment as well, of course :)

Related #1973 , #1586 , #1475
Might also benefit #524 #622

[felixdittrich92]

Hi @Razlaw 👋

Glad to see you back! 😊

I think this is more of a pre-processing topic - something of what’s done in:
https://github.com/fh2019ustc/DocTr-Plus

We already have a contrib module that was originally designed to be framework-independent. (Since the 1.0.0 release, that’s no longer relevant as TensorFlow support has been dropped and PyTorch is now our main backend)

The idea behind contrib is to offer an interface for functionalities like object detection, dewarping, and so on - designed to work standalone without being tightly coupled to the core logic of doctr.

https://github.com/mindee/doctr/tree/main/doctr/contrib

[Razlaw]

Sure :)

I tried these:

• https://github.com/tanguymagne/UVDoc
• https://github.com/cvlab-stonybrook/PaperEdge
• https://github.com/xiaomore/Document-Image-Dewarping
• https://github.com/ZZZHANG-jx/DocRes
  • This one is really nice for removing shadows btw
• https://github.com/lmmx/page-dewarp

[felixdittrich92]

Sure :)

I tried these:

• https://github.com/tanguymagne/UVDoc

• https://github.com/cvlab-stonybrook/PaperEdge

• https://github.com/xiaomore/Document-Image-Dewarping

• https://github.com/ZZZHANG-jx/DocRes

  • This one is really nice for removing shadows btw

• https://github.com/lmmx/page-dewarp

Thanks 🤗

[felixdittrich92]

@Razlaw One more seems to be the "newest": https://github.com/FelixHertlein/doc-matcher

[Razlaw]

Thanks for the link :)
If I got it right this method relies on a template the image can be matched with.
Unfortunately german receipts all look different.

[felixdittrich92]

If I understood it correct a template should be only required for the models with template in it's name:

Available models are:

identity
dewarpnet@inv3d
geotr@inv3d
geotr_template@inv3d
geotr_template_large@inv3d
docmatcher@inv3d