Set log level #3
Comments
|
@lucaro, can you please provide an example of such log? I don't see any in my case and I'm wondering if that is because your graph is taking a different path than mine. |
|
Sure, the following is just a small snippet from the produced output. The lines which start with date and time information are written to the standard error stream while the others are written to standard out. The Graph used while generating this output is this one: https://github.com/lucaro/VideoSaliencyFilter/blob/master/omcnn2clstm.pb
|
|
Ok, it looks to me that you are logging device placement. This is normally controlled in Java by setting the I would have expect the default value to be false, are you passing such proto at the construction of your sessions? |
|
Oops, the logDevicePlacement field was indeed set to true, must have accidentally autocompleted the wrong statement. After removing that statement, most, but not all of the log output disappears. The remiander looks like this:
The warnings look like they probably warrant their own issue at another time, but the rest looks like regular operation log, the level of which should be configurable somehow, right? |
|
That I personally don't know, never tried to do it before. The messages you are showing are at the I'll investigate a bit more how to turn those off but just to validate one last thing, are you in Android or just using the standard JDK? |
|
Looks like the protobuf warnings should go away if we use the latest protobuf release (see protocolbuffers/protobuf#3781). |
|
Thanks! Regarding my environment, I'm currently using OpenJDK 12. |
|
Hi @lucaro , sorry I didn’t came back sooner on this. After clearing device placement logs and upgrading protobuf version, is your execution still too verbose? |
|
The log output shown above (#3 (comment)) is after the removal of the device placement log flag. I'm using the most recent available tensorflow version mavencentral offers, which is 1.15. I did not explicitly add protobuf to the project dependencies, the full list of dependencies is this: https://github.com/lucaro/VideoSaliencyFilter/blob/master/build.gradle |
|
Hi @lucaro , I'm really sorry for the late reply, I wanted to come back on this issue as soon as I find some time, which only happens now... Digging in TF source code, I found out that there is an environment variable that set the minimum log level in the C++ core: https://github.com/tensorflow/tensorflow/blob/b82ab06d4b596a4aa964b2c78de3478e0cef6dd9/tensorflow/core/platform/default/logging.cc#L153 Can you set that environment variable |
|
I'm not entirely sure what causes it, but the amount of log output is different when I run the application from the IDE or from a jar. With the jar, the only remaining output is as follows, the environment variable you mentioned does not appear to make any difference to it.
|
|
OK so from the JAR, this kind of log seems normal to me (the Which IDE do you use? Are you sure the environment variable I've mentioned is not set somewhere in the execution settings of your app? |
|
It's not only the protobuf warning, the last line comes from TensorFlow. |
The line warning you that TF is not optimized to support your CPU instruction set has always been there so I don't see it as an issue. |
|
Well, at this point it's more of a philosophical question I guess but shouldn't the log output of a library, especially the one concerned with information rather than errors, be controllable by the main application using the library? |
|
We might be able to call the appropriate method in the guts of the C++ library to turn off the logger, but really the C++ library should be refactored a bit to expose better methods for controlling the logging behaviour. |
|
Hi @lucaro , I was able to turn the logs off by setting the configuration variable I gave you in my previous post. Here is my IntelliJ settings for running my app:
Output with value set to Output with value unset (or set to Please confirm that is works on you side as well and if it does, |
|
I can confirm that this setting changes the amount of log output. |
|
Thanks @lucaro,
do you still see "undesirable" messages in the logs after setting this? (i.e. anything that is just informative?) |
|
No, after setting this, only the protobuf warning remains. Since there is no way to set this from within the program though, this is only a solution for this particular instance and not for the log management in general. That would however rather be a feature request towards the C++ part of TensorFlow, right? |
|
Yes, I think we need to add an endpoint to set the value of this variable in a friendly matter, we’ll take a look at it. For the protobuf warning, I’ll come back to you as soon as I know for sure it is a C++ runtime issue, which is my feeling right now |
|
The IllegalReflectiveAccess warning is because the protobuf jar is messing with fields it shouldn't be. Its emitted on Java 9 and newer. Updating to a newer version will probably fix it. |
|
Oh, of course the warning to come from a Java artifact since it is about an illegal reflective access... But I can't reproduce it on my side, even if I include an additional dependency to @lucaro , are you sure that this warning does not come from another artifact that your application might depend on? |
|
TensorFlow is the only library I'm using in this particular project which has a dependency to protobuf. In case it helps, this is the complete list of dependencies: https://github.com/lucaro/VideoSaliencyFilter/blob/master/build.gradle |
|
Looks like it's fixed by using protobuf-java 3.7.0 or newer (https://github.com/protocolbuffers/protobuf/releases/tag/v3.7.0). The 1.15.0 proto artifact depends on protobuf-java 3.5.1. @karllessard, @sjamesr, where does the org.tensorflow:proto artifact come from? Is it one of the things we'll produce here, or is it produced by Google? |
|
Google has been doing the releases up to this point, I don't know exactly what the plan is for this artifact in the future. |
|
An new issue has been created for protobuf support in new TF Java: #21 |
Sync with master tensorflow on upstream
|
Bumping this issue, is there any way to set the log level within the Java Code now that the corresponding issue has been resolved? |
|
We spent some time looking into exporting TF logging messages into Java so they can be logged using a Java logging framework, but unfortunately that effort stalled as the TF log handler doesn't behave correctly - tensorflow/tensorflow#44995, #345. |
System information
Describe the current behavior
When using the current Java bindings for TensorFlow, the log gets filled with output from every operation when a model is evaluated. I was unable to find a way to set the log level via the Java API. Is this at all possible? If not, could you please consider adding this functionality in future versions?
The text was updated successfully, but these errors were encountered: