example AWS Glue pyspark job script that fetches all the catalog databases and tables metadata.
- first method uses spark sql
- second method uses python boto3 Glue client to interact with Glue API directly
--enable-glue-datacatalog is enabled for glue job to allow spark sql to access metadata catalog
Glue Console Script View
Twitter • Reddit