Laravel pdf parser module

build a module within my laravel application that allows uploading of pdf where specific texts are identified in the pdf and their xy coordinates are extracted

169d ago

I'll create a Laravel module for PDF text extraction with coordinates. This solution will allow users to upload PDFs and extract specific text along with their XY coordinates.

Installation and Setup Instructions

Install Required Dependencies:
composer require spatie/pdf-to-text
Install Poppler Utils (required for coordinate extraction):
- On Ubuntu/Debian:
  sudo apt-get install poppler-utils
- On macOS:
  brew install poppler
- On Windows, download binaries from poppler for Windows and add to your PATH
Run Migrations:
php artisan migrate
Configure Storage: Make sure your storage is properly configured and linked:
php artisan storage:link

How It Works

Upload PDF: Users upload a PDF file and specify search terms
Text Extraction: The system extracts text matching the search terms along with their XY coordinates
Storage: The extracted data is stored in the database
Visualization: The results page shows the extracted text with coordinates and provides a basic visualization

The module uses pdftotext from Poppler Utils with the -bbox option to get precise coordinates of text elements. If that fails, it falls back to a simpler extraction method using Spatie's PDF to text library.

Extending the Module

You can enhance this module by:

Adding more advanced PDF visualization using PDF.js
Implementing OCR capabilities for scanned documents using Tesseract
Adding batch processing for multiple PDFs
Creating export functionality for the extracted data (CSV, JSON)

This implementation provides a solid foundation that you can build upon based on your specific requirements.

169d ago

just tried it, but x-coordinate is always 0, the y-coordinate works

169d ago

I'll fix the issue with the x-coordinates always being 0. The problem is likely in how we're parsing the XML output from pdftotext or in the fallback method.

What Was Fixed

I've made several improvements to fix the issue with x-coordinates always being 0:

Enhanced XML Parsing: The updated code uses -bbox-layout instead of just -bbox for better coordinate extraction and properly handles the XML output.
Added Alternative Method: I've added a more robust alternative method that estimates x-coordinates based on character position in the text, which should work even if the primary method fails.
Improved Fallback Method: The fallback method now calculates x-coordinates based on character position rather than defaulting to 0.
Added Debug Logging: Added logging to help troubleshoot any remaining issues.
Enhanced Visualization: The PDF viewer now displays the actual coordinate values next to each highlighted area.

Additional Troubleshooting Steps

If you're still experiencing issues with the x-coordinates:

Check Poppler Version: Make sure you have a recent version of poppler-utils installed:
pdftotext --version
Verify PDF Format: Some PDFs (especially scanned ones) might not have proper text information. Try with a different PDF that was digitally created.
Check Debug Output: Look at the debug information section on the results page to see the raw coordinate data.
Try Direct Command: Run the pdftotext command directly to see its output:
pdftotext -bbox-layout your-pdf.pdf output.xml
Then examine output.xml to verify it contains proper x-coordinates.

Let me know if you're still having issues after these changes, and I can provide further assistance!

169d ago

resources

views

pdf-extractor

show.blade.php

WorkingWorking...

Working...

build a module within my laravel application that allows uploading of pdf where specific texts are identified in the pdf and their xy coordinates are extracted

169d ago

I'll create a Laravel module for PDF text extraction with coordinates. This solution will allow users to upload PDFs and extract specific text along with their XY coordinates.

Installation and Setup Instructions

Install Required Dependencies:
composer require spatie/pdf-to-text
Install Poppler Utils (required for coordinate extraction):
- On Ubuntu/Debian:
  sudo apt-get install poppler-utils
- On macOS:
  brew install poppler
- On Windows, download binaries from poppler for Windows and add to your PATH
Run Migrations:
php artisan migrate
Configure Storage: Make sure your storage is properly configured and linked:
php artisan storage:link

How It Works

Upload PDF: Users upload a PDF file and specify search terms
Text Extraction: The system extracts text matching the search terms along with their XY coordinates
Storage: The extracted data is stored in the database
Visualization: The results page shows the extracted text with coordinates and provides a basic visualization

Extending the Module

You can enhance this module by:

Adding more advanced PDF visualization using PDF.js
Implementing OCR capabilities for scanned documents using Tesseract
Adding batch processing for multiple PDFs
Creating export functionality for the extracted data (CSV, JSON)

This implementation provides a solid foundation that you can build upon based on your specific requirements.

just tried it, but x-coordinate is always 0, the y-coordinate works

169d ago

I'll fix the issue with the x-coordinates always being 0. The problem is likely in how we're parsing the XML output from pdftotext or in the fallback method.

What Was Fixed

I've made several improvements to fix the issue with x-coordinates always being 0:

Enhanced XML Parsing: The updated code uses -bbox-layout instead of just -bbox for better coordinate extraction and properly handles the XML output.
Added Alternative Method: I've added a more robust alternative method that estimates x-coordinates based on character position in the text, which should work even if the primary method fails.
Improved Fallback Method: The fallback method now calculates x-coordinates based on character position rather than defaulting to 0.
Added Debug Logging: Added logging to help troubleshoot any remaining issues.
Enhanced Visualization: The PDF viewer now displays the actual coordinate values next to each highlighted area.

Additional Troubleshooting Steps

If you're still experiencing issues with the x-coordinates:

Check Poppler Version: Make sure you have a recent version of poppler-utils installed:
pdftotext --version
Verify PDF Format: Some PDFs (especially scanned ones) might not have proper text information. Try with a different PDF that was digitally created.
Check Debug Output: Look at the debug information section on the results page to see the raw coordinate data.
Try Direct Command: Run the pdftotext command directly to see its output:
pdftotext -bbox-layout your-pdf.pdf output.xml
Then examine output.xml to verify it contains proper x-coordinates.

Let me know if you're still having issues after these changes, and I can provide further assistance!

169d ago

resources

views

pdf-extractor

show.blade.php

WorkingWorking...

Working...

PdfExtractorController.php

PdfTextExtractorService.php

2023_05_15_000001_create_pdfs_table.php

2023_05_15_000002_create_extracted_texts_table.php