Crawler series: using API

Application programming interface （Application Programming Interface, API） Usefulness ： It provides convenient and friendly interfaces for different applications . Different developers use different architectures , Even writing software in different languages has no problem —— because API The purpose of design is to become a communication language , Let different software share information .

Although different software applications have their own characteristics API, but “API” Often seen as “ Web applications API”, In general , Programmers can use HTTP Agreement to API Make a request to get some information ,API Will use XML(eXtensible Markup Language, Extensible markup language ) or JSON (Javascript Object Notation,Javascript Objects represent ) The format returns the information of the server response . Although most API Still in use XML, however JSON It is rapidly becoming the mainstream choice of data coding format .

Although this out of the box interface obtains pre packaged information , It seems that it has nothing to do with reptiles , But this view is only half right . Although most people don't usually use API As network data collection , But in fact, many of the technologies used by both （ It's all about sending HTTP request ） And the results （ It's all about getting information ） Not so much , The two are often complementary .

for example , You might put web crawlers and API Combine the information obtained , Because such information may be more meaningful .

## API summary

although API Not everywhere , But you can go through API Get a lot of useful information from . If you're interested in music , There is a song name 、 singer 、 Album Information API.

Here is API An example of a call ：

https://api.bigdatacloud.net/data/ip-geolocation-full?ip=27.30.84.181&localityLanguage=zh&key=bee73355d8ad4821a1c393c545e7f0

You enter the above URL through the browser , You can initiate a simple API call , give the result as follows ：

{

"ip": "199.21.99.90",

"localityLanguageRequested": "zh",

"isReachableGlobally": true,

"country": {

"isoAlpha2": "US",

"isoAlpha3": "USA",

"m49Code": 840,

"name": " The United States ",

"isoName": "United States of America (the)",

"isoNameFull": "the United States of America",

"isoAdminLanguages": [

{

"isoAlpha3": "eng",

"isoAlpha2": "en",

"isoName": "English",

"nativeName": "English"

}

],

"unRegion": "Europe and Northern America/Northern America",

"currency": {

"numericCode": 840,

"code": "USD",

"name": "US Dollar",

"minorUnits": 2

},

"wbRegion": {

"id": "NAC",

"iso2Code": "XU",

"value": "North America"

},

"wbIncomeLevel": {

"id": "HIC",

"iso2Code": "XD",

"value": "High income"

},

"callingCode": "1",

"countryFlagEmoji": "

Some of the results are excerpted above .

You might think , This is not to enter a web address in the browser window , Then press enter to get （ It's just JSON Format ） Information ？ Exactly? API What's the difference between calling and ordinary website access ？ If you don't think about API The name on the tall , In fact, there is no difference between the two .API Can pass HTTP Protocol Download File , and URL The protocol for accessing the website to obtain data is the same . It can realize all the things that are done online now .API The name API Why not call it a website , Actually, first of all API Have very strict grammar , secondly API use JSON or XML Format represents data , instead of HTML Format .

The above explains API Related concepts of , obtain API The data of , It's usually JSON data , Let's see Python How to parse in JSON data .

## analysis JSON data

Let's take the example just now , See how to pass API Acquired JSON Data to get the desired results ：

import requests

class ScrapeAPI(object):

def __init__(self):

self._api_url = 'https://api.bigdatacloud.net/data/ip-geolocation-full?ip=27.30.84.181&localityLanguage=zh&key=bee73355d8ad4821a1c19345e7f0'

def get_geolocation(self):

init_session = requests.Session()

response = init_session.get(self._api_url)

json_result = response.json()

get_country = json_result['country']['name']

get_location = json_result['location']

get_region = get_location['isoPrincipalSubdivision']

get_city = get_location['city']

get_locality_name = get_location['localityName']

area = f' At present IP Country ：{get_country}, region ：{get_region}, City ：{get_city}, Division ：{get_locality_name}'

print(area)

if __name__ == '__main__':

ScrapeAPI().get_geolocation()

The code above uses requests Library pair JSON Data analysis , Run the code to get the results we need .

That's what today is about API Related content .