min park

NLP and Data Analytics

Links for cURL to Requests

April 15, 2021NLP

cURL?

Shorthand of Client URL, cURL has wide-ranging purposes ranging from downloading remote packages (such as GitHub) to emulate REST API requests (GET / POST).

As it happens, handling the latter is tricky without resorting to cURL. In most cases, you don't get the same json-dict as shown in your browsers' preview.

cURL to Requests

First one is for the multi-purpose Requests library (requests library in Python)

https://curl.trillworks.com

In fact, this site provides bunch of programming languages you can convert to, not only Python or Node.

cURL to scrapy (Python)

scrapy

scrapy is one of the feature-rich scraping frameworks (as the name implies) available in Python. This is developed with Flask, so the usage is pretty much the same with other tools made with that same framework.

cURL to scrapy Request (curl2scrapy)

curl2scrapy

While the basic operation is similar to that using requests library, scrapy has slight different syntaxes and fetch command, which is to actually send request command to server and get the response.