Airflow has SimpleHttpOperator which can be used to invoke REST APIs. However using this operator is not exactly straightforward.
Airflow needs to be told about the connection parameters and all the other information that is needed to connect to external system. For this we need to create Connections. Open 'Connections' page through Admin->Connections link.
Expand the dropdown to see the various types of connection options available. For a REST call, create an HTTP connection. Give the host URL and any other details if required.
Now when we write our task using SimpleHttpOperator we will need to refer to the connection that was just created. The task below is making a post call to https://reqres.in/api/users API and passing it some data in JSON format.
myHttpTask = SimpleHttpOperator(
task_id='get_op',
method='POST',
http_conn_id='dcro',
data=json.dumps({
"name":"Morpheus",
"job":"Leader"
}),
endpoint='api/users',
headers={"Content-Type": "application/json"},
dag=dag)
Learn more about connections here: https://airflow.apache.org/docs/stable/howto/connection/index.html
Airflow needs to be told about the connection parameters and all the other information that is needed to connect to external system. For this we need to create Connections. Open 'Connections' page through Admin->Connections link.
Expand the dropdown to see the various types of connection options available. For a REST call, create an HTTP connection. Give the host URL and any other details if required.
Now when we write our task using SimpleHttpOperator we will need to refer to the connection that was just created. The task below is making a post call to https://reqres.in/api/users API and passing it some data in JSON format.
myHttpTask = SimpleHttpOperator(
task_id='get_op',
method='POST',
http_conn_id='dcro',
data=json.dumps({
"name":"Morpheus",
"job":"Leader"
}),
endpoint='api/users',
headers={"Content-Type": "application/json"},
dag=dag)
Learn more about connections here: https://airflow.apache.org/docs/stable/howto/connection/index.html
Comments