I'll create a Python program that fetches research papers from PubMed and identifies those with pharmaceutical/biotech company affiliations. Let's build this step by step.

Project Structure

First, let's set up the project structure with Poetry:

Now, let's create the pyproject.toml file for Poetry:

Let's create the core module for fetching papers:

Now, let's create the paper processor module:

Now, let's create the CLI module:

Let's create the package's __init__.py file:

Finally, let's create a README.md file:

Using pip

Usage

Command-line Options

• query: PubMed search query (supports full PubMed query syntax)
• -h, --help: Display usage instructions
• -d, --debug: Print debug information during execution
• -f, --file: Specify the filename to save the results (if not provided, print to console)
• -m, --max-results: Maximum number of results to fetch (default: 100)
• -e, --email: Email address to identify yourself to NCBI (required)
• -k, --api-key: NCBI API key for higher request limits

Code Organization

The project is organized as follows:

• pubmed_paper_fetcher/: Main package directory

  • __init__.py: Package initialization
  • cli.py: Command-line interface
  • pubmed_api.py: PubMed API client
  • paper_processor.py: Paper processing logic
• pyproject.toml: Poetry configuration
• README.md: Project documentation

How It Works

1. The tool queries PubMed using the Entrez API from Biopython
2. For each paper, it extracts author information and affiliations
3. It identifies non-academic affiliations using heuristics (keywords, patterns)
4. Papers with at least one author affiliated with a pharmaceutical/biotech company are included in the results
5. Results are formatted as a CSV file with the required columns

Tools and Libraries Used

• Poetry: Dependency management and packaging
• Biopython: Interface to NCBI Entrez API
• Requests: HTTP library for API calls
• tqdm: Progress bar for long-running operations
• Python typing: Type hints throughout the code

Development

Running Tests

Code Formatting

Type Checking

License

MIT

How to Use the Tool

1. Install the package using Poetry:

   git clone https://github.com/yourusername/pubmed-paper-fetcher.git cd pubmed-paper-fetcher poetry install poetry shell

2. Run the command-line tool:

   get-papers-list "cancer therapy" --email your.email@example.com --file results.csv

3. The tool will:

   • Search PubMed for papers matching your query
   • Identify papers with authors affiliated with pharmaceutical/biotech companies
   • Save the results to a CSV file with the required columns

Key Features

1. Comprehensive PubMed Search: Supports full PubMed query syntax
2. Smart Affiliation Detection: Uses heuristics to identify pharmaceutical and biotech company affiliations
3. Detailed Output: Provides comprehensive information including PubmedID, title, publication date, non-academic authors, company affiliations, and corresponding author email
4. Configurable: Command-line options for customizing search parameters and output
5. Rate Limiting: Implements proper rate limiting to comply with NCBI's API usage policies

Implementation Details

Identifying Non-Academic Affiliations

The tool uses a combination of techniques to identify non-academic affiliations:

1. Keyword Analysis: Looks for company-specific keywords (Inc, Ltd, GmbH, Pharma, Biotech, etc.)
2. Negative Matching: Excludes affiliations containing academic keywords (University, Hospital, Institute, etc.)
3. Pattern Recognition: Analyzes affiliation structure to identify company names

Error Handling

The tool implements robust error handling:

• API request failures
• Rate limiting and retries
• Invalid or missing data
• User interruptions

Performance Considerations

• Batch processing of PubMed IDs
• Progress tracking for long-running operations
• Efficient memory usage for large result sets

Bonus Implementation

As a bonus, the package is structured as a reusable module with a separate CLI interface. This allows it to be:

1. Imported and used programmatically in other Python applications
2. Run directly from the command line
3. Extended with additional functionality

To publish to test-pypi, run:

Future Improvements

Potential enhancements for future versions:

• Improved company name extraction using NLP techniques
• Support for additional metadata fields
• Integration with other bibliographic databases
• Enhanced filtering options
• Web interface or API endpoint

This tool provides researchers and analysts with an efficient way to identify research papers with pharmaceutical or biotech company affiliations, helping to track industry research activities and potential conflicts of interest.