How to suppress Spark logging?

[kchoi01]

The output is buried in the sea of log messages. Tried having a log4j.properties files in the project directory with spark log level set to WARN but doesn't seem to work.

log4j.logger.org.apache.spark=WARN

[makkenned]

What worked for me is copying the template log4j.properties to $SPARK_HOME/conf/, and editing

log4j.rootCategory=INFO, console

to read as

log4j.rootCategory=WARN, console

[kchoi01]

That probably works but I think there should be a way to have a per-application setting to override the default values?

[nkrishnaswami]

Less idiomatically, note that the println outputs are going to stdout, and
the log messages to stderr; so you can redirect them differently with
something like
$SPARK_HOME/bin/cmd line args 2>logs | tee output
Regards. N

On Sun, Nov 1, 2015 at 11:35 AM kchoi01 notifications@github.com wrote:

That probably works but I think there should be a way to have a
per-application setting to override the default values?

—
Reply to this email directly or view it on GitHub
#94 (comment)
.