IBM Support

OutOfMemoryError issue occurs after connecting to Flume 0.9.4 master and running "getconfigs" in Flume shell

Troubleshooting


Problem

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space will occur after connecting to Flume 0.9.4 master and running "getconfigs" in Flume shell.

Symptom

For example, when Flume master is running on bdvm070.svl.ibm.com

[biadmin@bdvm070 bin]$ ./flume shell -c bdvm070.svl.ibm.com:35871
2012-09-24 19:38:06,803 [main] INFO conf.FlumeConfiguration: Loading configurations from /opt/ibm/biginsights/flume/conf
Using default report port: 45678
Connecting to Flume master bdvm070.svl.ibm.com:35871:45678...
2012-09-24 19:38:07,066 [main] INFO util.AdminRPCThrift: Connected to master at bdvm070.svl.ibm.com:35871
==================================================
FlumeShell v0.9.4
Copyright (c) Cloudera 2010, All Rights Reserved
==================================================
Type a command to execute. Many commands only work when you are connected to a master node.

You might connect to a master node by typing:
connect host[:adminport=35873[:reportport=45678]]

[flume bdvm070.svl.ibm.com:35871:45678] getconfigs
JVMDUMP039I Processing dump event "systhrow", detail "java/lang/OutOfMemoryError" at 2012/09/24 19:38:19 - please wait.
JVMDUMP032I JVM requested Heap dump using '/opt/ibm/biginsights/flume/bin/heapdump.20120924.193819.29238.0001.phd' in response to an event
JVMDUMP010I Heap dump written to /opt/ibm/biginsights/flume/bin/heapdump.20120924.193819.29238.0001.phd
JVMDUMP032I JVM requested Java dump using '/opt/ibm/biginsights/flume/bin/javacore.20120924.193819.29238.0002.txt' in response to an event
JVMDUMP010I Java dump written to /opt/ibm/biginsights/flume/bin/javacore.20120924.193819.29238.0002.txt
JVMDUMP032I JVM requested Snap dump using '/opt/ibm/biginsights/flume/bin/Snap.20120924.193819.29238.0003.trc' in response to an event
JVMDUMP010I Snap dump written to /opt/ibm/biginsights/flume/bin/Snap.20120924.193819.29238.0003.trc
JVMDUMP013I Processed dump event "systhrow", detail "java/lang/OutOfMemoryError".
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
at org.apache.thrift.protocol.TBinaryProtocol.readStringBody(TBinaryProtocol.java:353)
at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:215)
at com.cloudera.flume.conf.thrift.FlumeMasterAdminServer$Client.recv_getConfigs(FlumeMasterAdminServer.java:263)
at com.cloudera.flume.conf.thrift.FlumeMasterAdminServer$Client.getConfigs(FlumeMasterAdminServer.java:249)
at com.cloudera.flume.util.AdminRPCThrift.getConfigs(AdminRPCThrift.java:71)
at com.cloudera.flume.util.FlumeShell.execCommand(FlumeShell.java:769)
at com.cloudera.flume.util.FlumeShell.executeLine(FlumeShell.java:834)
at com.cloudera.flume.util.FlumeShell.run(FlumeShell.java:892)
at com.cloudera.flume.util.FlumeShell.main(FlumeShell.java:978)

Resolving The Problem

This defect will not impact the main usage of Flume, which is to transfer data. You can also get node configuration from Flume master console. Because the Flume latest version (1.2.0) architecture has changed, there is no Flume master any more. This issue does not exist in Flume NG. This is a fixed upstream issue.

[{"Product":{"code":"SSCRJT","label":"IBM Db2 Big SQL"},"Business Unit":{"code":"BU059","label":"IBM Software w\/o TPS"},"Component":"Open Source Tools","Platform":[{"code":"PF016","label":"Linux"}],"Version":"2.0.0","Edition":"","Line of Business":{"code":"LOB10","label":"Data and AI"}}]

Document Information

Modified date:
18 July 2020

UID

swg21613435