Python stdin stderr logging
Logging in Python is quite straight forward, except for by the BasicConfig writes to
stderr and not to
This can cause
info logs to be written to
stderr and in AWS Glue it ends up in the error logs.
Setup logs to stdout
logging is a but complicated for some usecases.
print() in python writes to
stdout by default. But we can make it write to
AWS Glue Python Shell logs
You can use print statements in your Glue Python Shell job for logging. Glue captures
stderr by default.
There is no need to setup a logger in Glue Python Shell job.
Redirecting stderr and stdout
Most shells will show you
stderr logs in the console. You can redirect error logs and stdout logs into different files.
1> to redirect
2> to redirect
>> can be used to append operation