Skip to content

Conversation

mertemr
Copy link

@mertemr mertemr commented Aug 4, 2025

This PR migrates from synchronous requests to asynchronous httpx for making HTTP requests to PyPI, aiming to improve performance by enabling concurrent API calls when resolving package information.

  • Replaced requests with httpx for asynchronous HTTP requests
  • Added async/await pattern to get_imports_info and init functions
  • Implemented concurrent requests using asyncio.gather() for better performance

Transition to Asynchronous Code:

  • Refactored get_imports_info to an asynchronous function (async def) and split out _get_response as a helper function for making asynchronous HTTP requests using httpx. This allows concurrent fetching of package data from PyPI, improving performance.
  • Updated the init function to be asynchronous and used asyncio.run in the main function to execute it, enabling the integration of asynchronous workflows throughout the application. [1] [2]
  • Modified the call to get_imports_info in init to use await, ensuring proper handling of asynchronous operations.

Library Replacement:

  • Replaced the requests library with httpx for HTTP requests, enabling asynchronous request handling and modernizing the codebase.

@Copilot Copilot AI review requested due to automatic review settings August 4, 2025 06:05
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR migrates the HTTP request handling from synchronous requests to asynchronous httpx to improve performance when fetching package information from PyPI through concurrent API calls.

Key changes include:

  • Replaced requests library with httpx for asynchronous HTTP requests
  • Refactored get_imports_info to use async/await pattern with concurrent request handling
  • Updated the application flow to support asynchronous execution throughout

Reviewed Changes

Copilot reviewed 2 out of 3 changed files in this pull request and generated 2 comments.

File Description
pyproject.toml Added httpx dependency for asynchronous HTTP requests
pipreqs/pipreqs.py Migrated from synchronous requests to async httpx with concurrent request handling
Comments suppressed due to low confidence (2)

pyproject.toml:34

  • The httpx version 0.28.1 does not exist. The latest stable version of httpx is 0.27.x. Consider using a valid version range like "httpx>=0.27.0,<0.28.0".
    "httpx (>=0.28.1,<0.29.0)",

pipreqs/pipreqs.py:252

  • The variable name 'requests' is confusing as it shadows the previously used requests library name and doesn't clearly indicate it stores asyncio tasks. Consider renaming to 'tasks' or 'request_tasks'.
    requests = []


return data
elif response.status_code >= 300:
raise HTTPError(status_code=response.status_code, url=url)
Copy link

Copilot AI Aug 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The HTTPError constructor is being called with 'url' parameter, but the original yarg HTTPError exception likely expects 'reason' parameter instead. This could cause a TypeError at runtime.

Suggested change
raise HTTPError(status_code=response.status_code, url=url)
raise HTTPError(status_code=response.status_code, reason=f"Failed to fetch {url}")

Copilot uses AI. Check for mistakes.

)
result.append({"name": item, "version": data.latest_release_id})
return result
requests.append(asyncio.create_task(_get_response(client, f"{item}/json")))
Copy link

Copilot AI Aug 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The success logging message that was present in the original code (lines about 'Import named "%s" was resolved to "%s:%s"') has been removed, which reduces visibility into successful package resolutions for debugging purposes.

Suggested change
requests.append(asyncio.create_task(_get_response(client, f"{item}/json")))
requests.append(asyncio.create_task(_get_response(client, f"{item}/json", item)))

Copilot uses AI. Check for mistakes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant