Whenever we want to add in additional libraries then we build a custom image that include additional libraries/dependencies for example, we will add the Airflow’s provider for SQL Server. If you are new to Docker then feel free to check out the using custom images. The only thing that kind of bothers me is that apache-airflow-airflow-webserver-1 shows up as red in the docker logs. Let’s create a Dockerfile, this allows us to build a custom Airflow image. # and uncomment the "build" line below, Then run `docker-compose build` to build the images. # Comment the image line, place your Dockerfile in the directory where you placed the docker-compose.yaml
#Airflow docker file upgrade
# In order to add custom dependencies or upgrade provider packages you can use your extended image. However when I resolve the issue the code doesn't change and DAG remains broken.Īlso the following is my docker-compose file (I commented out airflow db init but may I should have kept it with the db upgrade parameter as true? My compose file is based on this template\ version: '3.1' Now however my dags are all broken and when I modify one without fixing the issue I can see see the updated line of code in the brief error information that shows up at the top of the webserver. So I just went ahead and created it and everything seemed to work fine. when I first used docker-compose to bring up my containers airflow was sending an error message saying that the column dag_has_import_errors doesn't exist. However I've been having some issues.Ī slight bit of background. This will give us access to the bash running in the container: airflow-on-docker-compose git: (master) docker-compose run webserver bash Starting airflow-on-docker-composepostgres1.
So in my case I've previously ran Airflow locally directly on my machine and now I'm trying to run it through containers using docker while also keeping the history of my previous dags. In order to run the tests in the environment we can just run: docker-compose run webserver bash.
0 kommentar(er)
