How to manage requests timeout?

[evemilano]

Hi all, I'm trying to download queries from a site with good traffic. By setting 16 months (480) days, the call times out.

I would like to understand if it was possible to raise the timeout or how to handle this exception.

Thanks for your help.

`timeout Traceback (most recent call last)
in
15
16 while not completed:
---> 17 ex = extract_gsc_data(webproperty, start_date, end_date, 'query', 'page')
18 if ex:
19 print("Success!")

16 frames
/usr/lib/python3.8/ssl.py in read(self, len, buffer)
1097 try:
1098 if buffer is not None:
-> 1099 return self._sslobj.read(len, buffer)
1100 else:
1101 return self._sslobj.read(len)

timeout: The read operation timed out`

[joshcarty]

Would you mind sharing the full error and the contents of your extract_gsc_data function?

[evemilano]

Hi @joshcarty thank you for the reply.

I pasted the full errore above, here it is:

Extracting data for <searchconsole.account.WebProperty(url='https://www.altalex.com/')>

timeout                                   Traceback (most recent call last)
[<ipython-input-82-a41475d1a853>](https://localhost:8080/#) in <module>
     14 
     15 while not completed:
---> 16     ex = extract_gsc_data(webproperty, start_date, end_date, 'query', 'page')
     17     if ex:
     18         print("Success!")

16 frames
[/usr/lib/python3.8/ssl.py](https://localhost:8080/#) in read(self, len, buffer)
   1097         try:
   1098             if buffer is not None:
-> 1099                 return self._sslobj.read(len, buffer)
   1100             else:
   1101                 return self._sslobj.read(len)

timeout: The read operation timed out

While the content of the script is:

giorni = 365
start_date, end_date = get_date_range(giorni)
completed = False
max_attempts = 3
attempts = 0

while not completed:
    ex = extract_gsc_data(webproperty, start_date, end_date, 'query', 'page')
    if ex:
        print("Success!")
        completed = True
    else:
        attempts += 1
        print(f"Attempt {attempts} of {max_attempts}")
        if attempts >= max_attempts:
            print("Error: maximum attempts reached.")
            break
        else:
            print("Error, retrying...")

I'm using the library in question to extract query-url of a site that generates a lot of traffic. I can't download beyond 60 days otherwise the timeout error I pasted in the first post will appear.

I was wondering if it was possible to tell the script to wait more, or to handle any timeout. Thanks for the support