Python requests get doesn't return anything - TagMerge
5Python requests get doesn't return anythingPython requests get doesn't return anything

Python requests get doesn't return anything

Asked 1 years ago
1
5 answers

Found the issue:

You need to have a User-Agent header:

import requests

if __name__ == '__main__':
    req = requests.Session()
    link12month = "https://search.codal.ir/api/search/v2/q?&Audited=true&AuditorRef=-1&Category=1&Childs=false&CompanyState=0&CompanyType=-1&Consolidatable=true&IsNotAudited=false&Isic=232007&Length=12&LetterCode=ن-10&LetterType=-1&Mains=true&NotAudited=false&NotConsolidatable=true&PageNumber=1&Publisher=false&Symbol=شپنا&TracingNo=-1&search=true"
    response = req.get(link12month, headers={'Accept': 'application/xml; charset=utf-8','User-Agent':'foo'})
    print(response.status_code, response.text)

You can also simplify your code a bit here as well:

import requests

if __name__ == '__main__':
    link12month = "https://search.codal.ir/api/search/v2/q?&Audited=true&AuditorRef=-1&Category=1&Childs=false&CompanyState=0&CompanyType=-1&Consolidatable=true&IsNotAudited=false&Isic=232007&Length=12&LetterCode=ن-10&LetterType=-1&Mains=true&NotAudited=false&NotConsolidatable=true&PageNumber=1&Publisher=false&Symbol=شپنا&TracingNo=-1&search=true"
    response = requests.get(link12month, headers={'Accept': 'application/xml; charset=utf-8','User-Agent':'foo'})
    print(response.status_code, response.text)

Make sure you verify if you can as well. If you want JSON, remove the 'Accept' header.

You can also simplify the URL a lot by adding the parameters as a dictionary, check out the docs here for the params argument https://www.w3schools.com/python/ref_requests_get.asp

Source: link

1

1.tried to get the url with firefox, got a valid response

2.tried to directly use requests library

import requests
uri = "https://search.codal.ir/api/search/v2/q?&Audited=true&AuditorRef=-1&Category=1&Childs=false&CompanyState=0&CompanyType=-1&Consolidatable=true&IsNotAudited=false&Isic=232007&Length=12&LetterCode=ن-10&LetterType=-1&Mains=true&NotAudited=false&NotConsolidatable=true&PageNumber=1&Publisher=false&Symbol=شپنا&TracingNo=-1&search=true"
session = requests.Session()
x = session.get(uri, timeout=2)

result: timeout exeption => server does not respond

3.tried to set header used by firefox

import requests
uri = "https://search.codal.ir/api/search/v2/q?&Audited=true&AuditorRef=-1&Category=1&Childs=false&CompanyState=0&CompanyType=-1&Consolidatable=true&IsNotAudited=false&Isic=232007&Length=12&LetterCode=ن-10&LetterType=-1&Mains=true&NotAudited=false&NotConsolidatable=true&PageNumber=1&Publisher=false&Symbol=شپنا&TracingNo=-1&search=true"

session = requests.Session()
session.headers.update({"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:96.0) Gecko/20100101 Firefox/96.0"})
response = session.get(uri, timeout=2)

print(response.content)

result looks like a valid xml response:

b'<SearchReportListDto xmlns:i="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://schemas.datacontract.org/2004/07/Codal.Services.Model.Dto.Search"><Letters><CodalLetterHeaderDto><AttachmentUrl>/Reports/Attachment.aspx?LetterSerial=cmjQJt%2bh5QxBpUMcT8rfPA%3d%3d</AttachmentUrl><CompanyName>\xd9\xbe\xd8\xa7\xd9\x84\xd8\xa7\xdb\x8c\xd8\xb4 \xd9\x86\xd9\x81\xd8\xaa \xd8\xa7\xd8\xb5\xd9\x81\xd9\x87\xd8\xa7\xd9\x86</CompanyName><ExcelUrl>https://excel.codal.ir/service/Excel/GetAll/cmjQJt%2bh5QxBpUMcT8rfPA%3d%3d/0</ExcelUrl><HasAttachment>true</HasAttachment><HasHtml>true</HasHtml><HasXbrl>false</HasXbrl><IsEstimate>false</IsEstimate><LetterCode>\xd9\x86-\xdb\xb1\xdb\xb0</LetterCode><PdfUrl>DownloadFile.aspx?hs=cmjQJt%2bh5QxBpUMcT8rfPA%3d%3d&amp;ft=1005&amp;let=6</PdfUrl><PublishDateTime>\xdb\xb1\xdb\xb4\xdb\xb0\xdb\xb0/\xdb\x...

Source: link

0

>>> import requests
>>> r = requests.get('https://api.github.com/events')
>>> r = requests.post('https://httpbin.org/post', data={'key': 'value'})
>>> r = requests.put('https://httpbin.org/put', data={'key': 'value'})
>>> r = requests.delete('https://httpbin.org/delete')
>>> r = requests.head('https://httpbin.org/get')
>>> r = requests.options('https://httpbin.org/get')
>>> payload = {'key1': 'value1', 'key2': 'value2'}
>>> r = requests.get('https://httpbin.org/get', params=payload)

Source: link

0

Let’s begin by installing the requests library. To do so, run the following command:
$ pip install requests
If you prefer to use Pipenv for managing Python packages, you can run the following:
$ pipenv install requests
Once requests is installed, you can use it in your application. Importing requests looks like this:
import requests
>>>
>>> requests.get('https://api.github.com')
<Response [200]>
>>>
>>> response = requests.get('https://api.github.com')

Source: link

0

Requests is a simple and elegant Python HTTP library. It provides methods for accessing Web resources via HTTP.
$ sudo service nginx start
version.py
#!/usr/bin/env python

import requests

print(requests.__version__)
print(requests.__copyright__)
The program prints the version and copyright of Requests.
$ ./version.py
2.21.0
Copyright 2018 Kenneth Reitz
read_webpage.py
#!/usr/bin/env python

import requests as req

resp = req.get("http://www.webcode.me")

print(resp.text)
The script grabs the content of the www.webcode.me web page.
resp = req.get("http://www.webcode.me")

Source: link

Recent Questions on python

    Programming Languages