由 ObjectInputStream 所引起的 Java 内存泄漏问题

Java 的 ObjectOutputStream 和 ObjectInputStream 各自保留一个对已发送/已接收对象的引用的列表。就是这些引用,会阻止垃圾收集器对这些对象内存的释放。
当新对象的数量不断增长时(比方说在服务器中),最终将抛出"Java.lang.OutOfMemoryError"。解决办法就是使用 writeUnshared() 和 readUnshared() 方法来取代 writeObject() 和 readObject() 方法。
介绍
怎样在 Java 中创建一个内存泄漏?这是一个很好地面试题,因为 Java 编程中由于底层的自动垃圾收集功能而无需担心内存释放问题。但在复杂的情况下你仍然会遭遇 Java.lang.OutOfMemoryError 问题。这里就描述了一种这样的情况:由 java.io.ObjectInputStream 和 java.io.ObjectOutputStream 所实现的 Java 序列化机制所引起的内存泄漏。
原因
ObjectOutputStream 和 ObjectInputStream 都各自维护了一个对其已发送/已接收对象的引用表。所以当 ObjectOutputStream 重新发送某对象时,可以仍发送该对象的句柄,相应的 ObjectInputStream 则将接收到的句柄转换为先前接收到的对象的引用。Java 文档中这样描述:

使用共享机制 (…) 对单个对象的多个引用进行编码。

可以说,Java 的这一 feature 减少了带宽和内存的使用量,但也仅适用于通过链接定期重新发送对象的程序中。但在通过链接发送新对象的情况下,此功能除了不必要的(不断增长的)引用表以外没有任何作用。
垃圾收集器的工作原理就是清理不再被引用的对象。但由于 ObjectInputStream 始终保持了对其所接收的每个对象的引用,它阻止了垃圾收集器对通过流所接收的任何对象的清理。就在诸如服务器之类的程序中,它们接收到越来越多的对象,最终内存将被耗尽,引发 OutOfMemoryError。
情况识别和分析
OOM 可能有很多种不同的原因所造成。为了能够确定 OOM 是上文所讨论的 ObjectInputStream 相关 feature 所造成的,我们首先得来分析该进程的 heap dump。有很多种办法可以针对一个 Java 进程生成一个 heap dump,也有很多种办法来分析这些 dump 文件。我们可以通过将 JVM 命令行选项添加到进程执行命令行来生成 dump。还可以针对一个已经在运行中的程序来生成一个 dump。其中,通过添加命令行选项的方式可以在有 OOM 错误时生成 dump 文件(当然,还有一些其他命令行选项通过信号控制方式来在任意时间生成 dump,不管程序状态如何)。命令行选项如下:

-XX:+HeapDumpOnOutOfMemoryError

在 OOM 抛出时,一个名为 java_pid.hprof 的文件将会在该进程的执行目录所生成。有 n 多 dump 分析程序可以对 dump 文件进行分析,本文只对 jvm 原生自带的 VisualVM 进行介绍。
在 VisualVM 中,打开你收集到的 hprof 文件。在 classes 选项卡中,可以找到所有的类,这些类默认是按照 dump 中所存在的它们所拥有的实例数进行排序。当然,顶部的类极可能就是实例数暴多且导致内存错误的那个类。按字母顺序对类列表进行排序也是很有帮助的,这样可以将同一包的类分组在一起,进而可以对它们进行比较。
双击该列表中的一个类,视图切换至 instances 选项卡。对于每个实例,都有两个窗口。下边的那个保存指向该实例的参考路径。每个路径的边缘是一个垃圾收集 root(由三角形标记)。这样每个边缘点都是一个对象,该对象持有一个引用,防止 GC 对该对象的清理。如果很多这种爆炸的类实例都指向了一个 ObjectInputStream 对象,那么导致 OOM 的罪魁祸首可能就是本文所讨论的主题。
解决方案
首先,不要做的事情:尝试让 GC 更好的工作是没有好处的,比如调用

System.gc()

命令。OOM 事件的先决条件就是 GC 已经尽其最大努力分配了所需要的内存,但并未成功分配。
为了使对象流不再使用上文讨论的引用机制,I/O 方法:

writeObject()
readObject()

需要替换为以下方法:

writeUnshared()
readUnshared()

不同于 readObject() 方法的是,readUnshared() 方法不会在 ObjectInputStream 的引用列表中保存引用,因此能够防止内存泄漏。需要注意的是,如果传输对象已经实现了来自 Serializable 接口的 readResolve() 方法,则该对象的引用可能还是会被传递并保留在引用列表中。可以在 VisualVM 中对该引用的持有者进行进一步分析。
还有一种解决方案是在每次写入后调用 ObjectOutputStream reset() 方法。这将具有相同的效果。
关于使用 C vs. Java 所开发服务器的讨论
本文所讨论的内存泄漏很难被预料到。具有自动垃圾收集功能的语言进行开发的本质是,开发人员无需投入精力来了解处于不同状态的程序的内存状态。当然,这也是自动垃圾收集的优势之一。使用自动决策算法的陷阱是其决策必须保守。当关注 GC 时,这意味着在不确定是否需要清除内存时,它将始终选择不清除可能不需要的内存,而不会冒清除所需内存的风险。这就是为什么每个引用对象都不会被清除的原因,尽管程序不会使用所有引用对象。在由 ObjectInputStream 引起泄漏的情况下,对于开发人员来说,引用是完全隐藏和意外的。导致内存泄漏的其他情况可能是,例如,当保存对静态对象的引用数量不断增长时(永远不会清除自身),或者在传递内部类对象时(保留对外部类的隐式引用)。这些示例对开发人员来说更为明确,但泄漏被提前发现的机会仍然很小。
另一种选择是不使用自动垃圾收集,而是让开发人员自己处理所有内存,就像在 C 和 C++ 中进行开发时所做的那样。这种开发需要开发人员做更多的工作,但是对程序的内存状态的理解水平要高得多。它虽然也不能防止内存错误的发生,但是可以使开发人员在分析它们时更了解发生了什么。
C 语言和 Java 语言对开发人员对内存责任的不同态度的另一个例子是,在 C 网络程序中,总是很清楚通过链接发送的每个数据段的大小,而不是 Java 的 Object 流那样对其中完全不清楚。在 C 语言中,开发人员甚至还必须考虑操作系统对数据结构进行的地址对齐(使用 attribute((packed)))。
大多数服务器程序都是用 Java 或 Python 等带有自动垃圾收集器的语言所编写的。对于大部分人来讲,节省开发人员的时间比这里提到的陷阱更有价值。
原文链接:Java memory leak caused by ObjectInputStream

已标记关键词 清除标记
软件环境: aix 6.1+weblogic10.3+oracle11 系统每天同时在线人数100左右,数据库的数据量有多张频繁操作的表数据记录在千万以上,主要后台自动处理线程过多。 系统前段时间运行一直正常,只是从7月底出现OOM,最近差不多半个月出现一次。 对weblogic产生的phd文件和javacore文件分析的不太清楚。。。 以下是javacore的日志: ***WARNING*** Java heap is almost exhausted : 0% free Java heap Please enable verbosegc trace and use IBM Pattern Modeling and Analysis Tool(http://www.alphaworks.ibm.com/tech/pmat) to analyze garbage collection activities. If heapdumps are generated at the same time, please use IBM HeapAnalyzer(http://www.alphaworks.ibm.com/tech/heapanalyzer) to analyze Java heap. File name : C:\Users\Wcy\Desktop\20140813\ebills\heapdump\javacore.20140813.091552.10551312.0007.txt Cause of thread dump : Dump Event "systhrow" (00040000) Detail "java/lang/OutOfMemoryError" received Date: 2014/08/13 at 09:16:49 Process ID : Not available Operating System : AIX 6.1 Processor Architecture : ppc Number of Processors : 12 Java version : JRE 1.6.0 IBM J9 2.4 AIX ppc-32 build jvmap3260sr9-20110624_85526 Virtual machine version : VM build 20110624_085526 Just-In-Time(JIT) compiler switch, Ahead-Of-Time (AOT) compiler switch, Compiler version : JIT enabled, AOT enabled - r9_20101028_17488ifx17 Garbage collector version : GC - 20101027_AA Java Heap Information Maximum Java heap size : 1024m Initial Java heap size : 512m Java Home Directory : /usr/java6/jre Java DLL Directory : /usr/java6/jre/bin System Classpath : /usr/java6/jre/lib/vm.jar;/usr/java6/jre/lib/annotation.jar;/usr/java6/jre/lib/beans.jar;/usr/java6/jre/lib/java.util.jar;/usr/java6/jre/lib/jndi.jar;/usr/java6/jre/lib/logging.jar;/usr/java6/jre/lib/security.jar;/usr/java6/jre/lib/sql.jar;/usr/java6/jre/lib/ibmorb.jar;/usr/java6/jre/lib/ibmorbapi.jar;/usr/java6/jre/lib/ibmcfw.jar;/usr/java6/jre/lib/rt.jar;/usr/java6/jre/lib/charsets.jar;/usr/java6/jre/lib/resources.jar;/usr/java6/jre/lib/ibmpkcs.jar;/usr/java6/jre/lib/ibmcertpathfw.jar;/usr/java6/jre/lib/ibmjgssfw.jar;/usr/java6/jre/lib/ibmjssefw.jar;/usr/java6/jre/lib/ibmsaslfw.jar;/usr/java6/jre/lib/ibmjcefw.jar;/usr/java6/jre/lib/ibmjgssprovider.jar;/usr/java6/jre/lib/ibmjsseprovider2.jar;/usr/java6/jre/lib/ibmcertpathprovider.jar;/usr/java6/jre/lib/ibmxmlcrypto.jar;/usr/java6/jre/lib/management-agent.jar;/usr/java6/jre/lib/xml.jar;/usr/java6/jre/lib/jlm.jar;/usr/java6/jre/lib/javascript.jar; User Arguments : -Xjcl:jclscar_24 -Dcom.ibm.oti.vm.bootstrap.library.path=/usr/java6/jre/lib/ppc -Dsun.boot.library.path=/usr/java6/jre/lib/ppc -Djava.library.path=/usr/java6/jre/lib/ppc:/usr/java6/jre/lib/ppc:/usr/java6/jre/lib/ppc/j9vm:/usr/java6/jre/lib/ppc/j9vm:/usr/java6/jre/lib/ppc:/usr/java6/jre/../lib/ppc::/home/ebills/bea/wlserver_10.3/server/native/aix/ppc:/usr/lib:/usr/lib -Djava.home=/usr/java6/jre -Djava.ext.dirs=/usr/java6/jre/lib/ext -Duser.dir=/weblogic/ebills/bea/user_projects/domains/nbdomain _j2se_j9=71168 0xF0A89414 -Djava.runtime.version=pap3260sr9fp2-20110627_03 (SR9 FP2) -Xdump -Djava.class.path=:/home/ebills/bea/patch_wls1030/profiles/default/sys_manifest_classpath/weblogic_patch.jar:/home/ebills/bea/patch_cie660/profiles/default/sys_manifest_classpath/weblogic_patch.jar:/usr/java6/lib/tools.jar:/home/ebills/bea/wlserver_10.3/server/lib/weblogic_sp.jar:/home/ebills/bea/wlserver_10.3/server/lib/weblogic.jar:/home/ebills/bea/modules/features/weblogic.server.modules_10.3.0.0.jar:/home/ebills/bea/wlserver_10.3/server/lib/webservices.jar:/home/ebills/bea/modules/org.apache.ant_1.6.5/lib/ant-all.jar:/home/ebills/bea/modules/net.sf.antcontrib_1.0.0.0_1-0b2/lib/ant-contrib.jar::/home/ebills/bea/wlserver_10.3/common/eval/pointbase/lib/pbclient57.jar:/home/ebills/bea/wlserver_10.3/server/lib/xqrl.jar:: -Xms512m -Xmx1024m -da -Dplatform.home=/home/ebills/bea/wlserver_10.3 -Dwls.home=/home/ebills/bea/wlserver_10.3/server -Dweblogic.home=/home/ebills/bea/wlserver_10.3/server -Dweblogic.management.discover=true -Dwlw.iterativeDev=false -Dwlw.testConsole=false -Dwlw.logErrorsToConsole= -Dclient.encoding.override=GBK -Dfile.encoding=GBK -Duser.language=zh -Duser.region=CN -Ddefault.client.encoding=GBK -Dweblogic.threadpool.MinPoolSize=200 -Dweblogic.threadpool.MaxPoolSize=500 -Djava.awt.headless=true -Dweblogic.ext.dirs=/home/ebills/bea/patch_wls1030/profiles/default/sysext_manifest_classpath:/home/ebills/bea/patch_cie660/profiles/default/sysext_manifest_classpath -Dweblogic.management.username=weblogic -Dweblogic.management.password=weblogic -Dweblogic.Name=AdminServer -Djava.security.policy=/home/ebills/bea/wlserver_10.3/server/lib/weblogic.policy -Dsun.java.command=weblogic.Server -Dsun.java.launcher=SUN_STANDARD _port_library 0xF0A89C18 _org.apache.harmony.vmi.portlib 0x3013DE18 User Limit Analysis User Limit Analysis Type Soft Limit Hard Limit RLIMIT_AS unlimited unlimited RLIMIT_CORE unlimited unlimited RLIMIT_CPU unlimited unlimited RLIMIT_DATA 2,147,483,645 bytes unlimited RLIMIT_FSIZE unlimited unlimited RLIMIT_NOFILE 1,024 1,024 RLIMIT_RSS unlimited unlimited RLIMIT_STACK 2,147,483,646 bytes 2,147,483,646 bytes Environment Variables Analysis Environment Variable Environment Variable Value _ /usr/java6/bin/java POST_CLASSPATH :/home/ebills/bea/wlserver_10.3/common/eval/pointbase/lib/pbclient57.jar:/home/ebills/bea/wlserver_10.3/server/lib/xqrl.jar CLUSTER_PROPERTIES -Dweblogic.management.discover=true JAVA_VENDOR IBM LANG Zh_CN PRODUCTION_MODE true DOMAIN_HOME /weblogic/ebills/bea/user_projects/domains/nbdomain LOGIN ebills CLASSPATHSEP : DATABASE_CLASSPATH /home/ebills/bea/wlserver_10.3/common/eval/pointbase/lib/pbclient57.jar CIE660_PATCH_LIBPATH /home/ebills/bea/patch_cie660/profiles/default/native WLS1030_PATCH_EXT_DIR /home/ebills/bea/patch_wls1030/profiles/default/sysext_manifest_classpath POINTBASE_HOME /home/ebills/bea/wlserver_10.3/common/eval/pointbase SSH_TTY /dev/pts/0 debugFlag false MEM_MAX_PERM_SIZE -XX:MaxPermSize=1024m SUN_JAVA_HOME CLCMD_PASSTHRU 1 PATCH_CLASSPATH /home/ebills/bea/patch_wls1030/profiles/default/sys_manifest_classpath/weblogic_patch.jar:/home/ebills/bea/patch_cie660/profiles/default/sys_manifest_classpath/weblogic_patch.jar PATH /home/ebills/bea/wlserver_10.3/server/bin:/home/ebills/bea/modules/org.apache.ant_1.6.5/bin:/usr/java6/jre/bin:/usr/java6/bin:/usr/bin:/etc:/usr/sbin:/usr/ucb:/home/ebills/bin:/usr/bin/X11:/sbin:/usr/java6/bin:/usr/java6/jre/bin:. FEATURES_DIR /home/ebills/bea/modules/features CIE660_PATCH_PATH /home/ebills/bea/patch_cie660/profiles/default/native verboseLoggingFlag false ANT_CONTRIB /home/ebills/bea/modules/net.sf.antcontrib_1.0.0.0_1-0b2 PATCH_PATH /home/ebills/bea/patch_wls1030/profiles/default/native:/home/ebills/bea/patch_cie660/profiles/default/native BEA_JAVA_HOME JAVA_VM ARDIR /home/ebills/bea/wlserver_10.3/server/lib LC__FASTMSG true POINTBASE_CLASSPATH :/home/ebills/bea/wlserver_10.3/common/eval/pointbase/lib/pbembedded57.jar:/home/ebills/bea/wlserver_10.3/common/eval/pointbase/lib/pbclient57.jar SSH_AUTH_SOCK /tmp/ssh-IbX9437420/agent.9437420 CIE660_PATCH_CLASSPATH /home/ebills/bea/patch_cie660/profiles/default/sys_manifest_classpath/weblogic_patch.jar JAVA_PROFILE JAVA_DEBUG CLASSPATH :/home/ebills/bea/patch_wls1030/profiles/default/sys_manifest_classpath/weblogic_patch.jar:/home/ebills/bea/patch_cie660/profiles/default/sys_manifest_classpath/weblogic_patch.jar:/usr/java6/lib/tools.jar:/home/ebills/bea/wlserver_10.3/server/lib/weblogic_sp.jar:/home/ebills/bea/wlserver_10.3/server/lib/weblogic.jar:/home/ebills/bea/modules/features/weblogic.server.modules_10.3.0.0.jar:/home/ebills/bea/wlserver_10.3/server/lib/webservices.jar:/home/ebills/bea/modules/org.apache.ant_1.6.5/lib/ant-all.jar:/home/ebills/bea/modules/net.sf.antcontrib_1.0.0.0_1-0b2/lib/ant-contrib.jar::/home/ebills/bea/wlserver_10.3/common/eval/pointbase/lib/pbclient57.jar:/home/ebills/bea/wlserver_10.3/server/lib/xqrl.jar:: LOGNAME ebills SAMPLES_HOME /home/ebills/bea/wlserver_10.3/samples MAIL /usr/spool/mail/ebills enableHotswapFlag POINTBASE_TOOLS /home/ebills/bea/wlserver_10.3/common/eval/pointbase/lib/pbtools57.jar LOCPATH /usr/lib/nls/loc MODULES_DIR /home/ebills/bea/modules PATCH_LIBPATH /home/ebills/bea/patch_wls1030/profiles/default/native:/home/ebills/bea/patch_cie660/profiles/default/native PATHSEP : WLS1030_PATCH_PATH /home/ebills/bea/patch_wls1030/profiles/default/native WLS1030_PATCH_CLASSPATH /home/ebills/bea/patch_wls1030/profiles/default/sys_manifest_classpath/weblogic_patch.jar iterativeDevFlag false doExitFlag false USER ebills SERVER_NAME AdminServer AUTHSTATE compat SERVER_CLASS weblogic.Server PRE_CLASSPATH JAVA_PROPERTIES -Dplatform.home=/home/ebills/bea/wlserver_10.3 -Dwls.home=/home/ebills/bea/wlserver_10.3/server -Dweblogic.home=/home/ebills/bea/wlserver_10.3/server -Dweblogic.management.discover=true BEA_HOME /home/ebills/bea MEM_DEV_ARGS SHELL /usr/bin/ksh ODMDIR /etc/objrepos OMNIORB_CONFIG /home/gjyw/ebills/config/ecorba.cfg JAVA_HOME /usr/java6 testConsoleFlag false WLS_HOME /home/ebills/bea/wlserver_10.3/server WLS1030_PATCH_LIBPATH /home/ebills/bea/patch_wls1030/profiles/default/native CIE660_PATCH_EXT_DIR /home/ebills/bea/patch_cie660/profiles/default/sysext_manifest_classpath JAVA_OPTIONS -da -Dplatform.home=/home/ebills/bea/wlserver_10.3 -Dwls.home=/home/ebills/bea/wlserver_10.3/server -Dweblogic.home=/home/ebills/bea/wlserver_10.3/server -Dweblogic.management.discover=true -Dwlw.iterativeDev=false -Dwlw.testConsole=false -Dwlw.logErrorsToConsole= -Dclient.encoding.override=GBK -Dfile.encoding=GBK -Duser.language=zh -Duser.region=CN -Ddefault.client.encoding=GBK -Dweblogic.threadpool.MinPoolSize=200 -Dweblogic.threadpool.MaxPoolSize=500 -Djava.awt.headless=true -Dweblogic.ext.dirs=/home/ebills/bea/patch_wls1030/profiles/default/sysext_manifest_classpath:/home/ebills/bea/patch_cie660/profiles/default/sysext_manifest_classpath -Dweblogic.management.username=weblogic -Dweblogic.management.password=weblogic HOME /home/ebills MEM_ARGS -Xms512m -Xmx1024m WEBLOGIC_EXTENSION_DIRS /home/ebills/bea/patch_wls1030/profiles/default/sysext_manifest_classpath:/home/ebills/bea/patch_cie660/profiles/default/sysext_manifest_classpath ANT_HOME /home/ebills/bea/modules/org.apache.ant_1.6.5 FILEDIR /home/gjyw/ebills/acct SSH_CONNECTION 10.100.67.22 3456 10.100.133.2 22 SSH_CLIENT 10.100.67.22 3456 22 WEBLOGIC_CLASSPATH /home/ebills/bea/patch_wls1030/profiles/default/sys_manifest_classpath/weblogic_patch.jar:/home/ebills/bea/patch_cie660/profiles/default/sys_manifest_classpath/weblogic_patch.jar:/usr/java6/lib/tools.jar:/home/ebills/bea/wlserver_10.3/server/lib/weblogic_sp.jar:/home/ebills/bea/wlserver_10.3/server/lib/weblogic.jar:/home/ebills/bea/modules/features/weblogic.server.modules_10.3.0.0.jar:/home/ebills/bea/wlserver_10.3/server/lib/webservices.jar:/home/ebills/bea/modules/org.apache.ant_1.6.5/lib/ant-all.jar:/home/ebills/bea/modules/net.sf.antcontrib_1.0.0.0_1-0b2/lib/ant-contrib.jar TERM vt100 MAILMSG [YOU HAVE NEW MAIL] POINTBASE_FLAG false LONG_DOMAIN_HOME /weblogic/ebills/bea/user_projects/domains/nbdomain PWD /weblogic/ebills/bea/user_projects/domains/nbdomain TZ Asia/Shanghai MEM_PERM_SIZE -XX:PermSize=512m BZJFILEDIR /weblogic/ebills/gjyw/ebills/ebillsData/DepositAcct WL_HOME /home/ebills/bea/wlserver_10.3 DEBUG_PORT 8453 A__z ! LOGNAME IBM_JVM_AIXTHREAD_SCOPE_NEW_VALUE S AIXTHREAD_SCOPE S IBM_JVM_CHANGED_ENVVARS_10551312 AIXTHREAD_SCOPE,NULLPTR,CORE_MMAP,LDR_CNTRL IBM_JVM_NULLPTR_NEW_VALUE NOSEGV NULLPTR NOSEGV IBM_JVM_CORE_MMAP_NEW_VALUE yes CORE_MMAP yes IBM_JVM_LDR_CNTRL_NEW_VALUE MAXDATA=0XA0000000@DSA LDR_CNTRL MAXDATA=0XA0000000@DSA NLSPATH /usr/lib/nls/msg/%L/%N:/usr/lib/nls/msg/%L/%N.cat LIBPATH /usr/java6/jre/lib/ppc:/usr/java6/jre/lib/ppc/j9vm:/usr/java6/jre/lib/ppc/j9vm:/usr/java6/jre/lib/ppc:/usr/java6/jre/../lib/ppc::/home/ebills/bea/wlserver_10.3/server/native/aix/ppc:/usr/lib:/usr/java6/jre/lib/ppc/headless IBM_JAVA_COMMAND_LINE /usr/java6/bin/java -Xms512m -Xmx1024m -da -Dplatform.home=/home/ebills/bea/wlserver_10.3 -Dwls.home=/home/ebills/bea/wlserver_10.3/server -Dweblogic.home=/home/ebills/bea/wlserver_10.3/server -Dweblogic.management.discover=true -Dwlw.iterativeDev=false -Dwlw.testConsole=false -Dwlw.logErrorsToConsole= -Dclient.encoding.override=GBK -Dfile.encoding=GBK -Duser.language=zh -Duser.region=CN -Ddefault.client.encoding=GBK -Dweblogic.threadpool.MinPoolSize=200 -Dweblogic.threadpool.MaxPoolSize=500 -Djava.awt.headless=true -Dweblogic.ext.dirs=/home/ebills/bea/patch_wls1030/profiles/default/sysext_manifest_classpath:/home/ebills/bea/patch_cie660/profiles/default/sysext_manifest_classpath -Dweblogic.management.username=weblogic -Dweblogic.management.password=weblogic -Dweblogic.Name=AdminServer -Djava.security.policy=/home/ebills/bea/wlserver_10.3/server/lib/weblogic.policy weblogic.Server Free Java heap size: 11.89 KB Allocated Java heap size: 1 GB Memory Segment Analysis Memory Segment Analysis Memory Type # of Segments Used Memory(bytes) Used Memory(%) Free Memory(bytes) Free Memory(%) Total Memory(bytes) Internal 431 0 0 28,332,996 100 28,332,996 Object(reserved) 1 1,073,741,824 100 0 0 1,073,741,824 Class 17,984 174,683,076 96.44 6,441,464 3.56 181,124,540 JIT Code Cache 7 58,720,256 100 0 0 58,720,256 JIT Data Cache 4 26,811,512 79.9 6,742,920 20.1 33,554,432 Overall 18,427 1,333,956,668 96.98 41,517,380 3.02 1,375,474,048 Current Thread : Thread Name [ACTIVE] ExecuteThread: '119' for queue: 'weblogic.kernel.Default (self-tuning)' State Runnable Java Stack at java/lang/String. (String.java:350(Compiled Code)) at java/lang/Throwable.printStackTrace(Throwable.java:369(Compiled Code)) at java/lang/Throwable.printStackTrace(Throwable.java:212(Compiled Code)) at weblogic/utils/StackTraceUtilsClient.throwable2StackTrace(StackTraceUtilsClient.java:25(Compiled Code)) at weblogic/jdbc/common/internal/ConnectionEnv.setup(ConnectionEnv.java:308(Compiled Code)) at weblogic/common/resourcepool/ResourcePoolImpl.reserveResource(ResourcePoolImpl.java:303(Compiled Code)) at weblogic/jdbc/common/internal/ConnectionPool.reserve(ConnectionPool.java:427(Compiled Code)) at weblogic/jdbc/common/internal/ConnectionPool.reserve(ConnectionPool.java:316(Compiled Code)) at weblogic/jdbc/common/internal/ConnectionPoolManager.reserve(ConnectionPoolManager.java:85(Compiled Code)) at weblogic/jdbc/common/internal/ConnectionPoolManager.reserve(ConnectionPoolManager.java:61(Compiled Code)) at weblogic/jdbc/jta/DataSource.getXAConnectionFromPool(DataSource.java:1450(Compiled Code)) at weblogic/jdbc/jta/DataSource.refreshXAConnAndEnlist(DataSource.java:1272(Compiled Code)) at weblogic/jdbc/jta/DataSource.getConnection(DataSource.java:425(Compiled Code)) at weblogic/jdbc/jta/DataSource.connect(DataSource.java:382(Compiled Code)) at weblogic/jdbc/common/internal/RmiDataSource.getConnection(RmiDataSource.java:336(Compiled Code)) at com/amerisia/ebills/commons/util/ServiceLocator.getConnection(ServiceLocator.java:66(Compiled Code)) at com/amerisia/ebills/commons/util/BaseDAO.makeConnection(BaseDAO.java:72(Compiled Code)) at com/amerisia/ebills/commons/util/BaseDAO.loadAllRowBySql(BaseDAO.java:623(Compiled Code)) at com/amerisia/ebills/chat/ejbs/ChatDAO.getLeaveWord(ChatDAO.java:119(Compiled Code)) at com/amerisia/ebills/chat/ejbs/ChatManagerEJB.getLeaveWord(ChatManagerEJB.java:165(Compiled Code)) at com/amerisia/ebills/chat/ejbs/ChatManager_oxcdb1_ELOImpl.getLeaveWord(ChatManager_oxcdb1_ELOImpl.java:358(Compiled Code)) at com/amerisia/ebills/chat/facade/ChatNoteManagerEJB.getLeaveWord(ChatNoteManagerEJB.java:110(Compiled Code)) at com/amerisia/ebills/chat/facade/ChatNoteManager_hfunq5_EOImpl.getLeaveWord(ChatNoteManager_hfunq5_EOImpl.java:339(Compiled Code)) at com/amerisia/ebills/chat/facade/ChatNoteManager_hfunq5_EOImpl_WLSkel.invoke(Bytecode PC:240(Compiled Code)) at weblogic/rmi/internal/ServerRequest.sendReceive(ServerRequest.java:174(Compiled Code)) at weblogic/rmi/cluster/ClusterableRemoteRef.invoke(ClusterableRemoteRef.java:345(Compiled Code)) at weblogic/rmi/cluster/ClusterableRemoteRef.invoke(ClusterableRemoteRef.java:259(Compiled Code)) at com/amerisia/ebills/chat/facade/ChatNoteManager_hfunq5_EOImpl_1030_WLStub.getLeaveWord(Bytecode PC:37(Compiled Code)) at com/amerisia/ebills/chat/action/ChatAction.getMsgsForLeft(ChatAction.java:466(Compiled Code)) at sun/reflect/GeneratedMethodAccessor234.invoke(Bytecode PC:64(Compiled Code)) at sun/reflect/DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:37(Compiled Code)) at java/lang/reflect/Method.invoke(Method.java:589(Compiled Code)) at com/amerisia/ebills/commons/action/EbillsBaseDispatchAction.dispatchMethod(EbillsBaseDispatchAction.java:1072(Compiled Code)) at org/apache/struts/actions/DispatchAction.execute(DispatchAction.java:194(Compiled Code)) at org/apache/struts/action/RequestProcessor.processActionPerform(RequestProcessor.java(Compiled Code)) at org/apache/struts/action/RequestProcessor.process(RequestProcessor.java:203(Compiled Code)) at org/apache/struts/action/ActionServlet.process(ActionServlet.java:1196(Compiled Code)) at org/apache/struts/action/ActionServlet.doGet(ActionServlet.java:414(Compiled Code)) at javax/servlet/http/HttpServlet.service(HttpServlet.java:707(Compiled Code)) at javax/servlet/http/HttpServlet.service(HttpServlet.java:820(Compiled Code)) at weblogic/servlet/internal/StubSecurityHelper$ServletServiceAction.run(StubSecurityHelper.java:227(Compiled Code)) at weblogic/servlet/internal/StubSecurityHelper.invokeServlet(StubSecurityHelper.java:121(Compiled Code)) at weblogic/servlet/internal/ServletStubImpl.execute(ServletStubImpl.java:292(Compiled Code)) at weblogic/servlet/internal/TailFilter.doFilter(TailFilter.java:26(Compiled Code)) at weblogic/servlet/internal/FilterChainImpl.doFilter(FilterChainImpl.java:42(Compiled Code)) at com/amerisia/ebills/commons/web/EncodingFilter.doFilter(EncodingFilter.java(Compiled Code)) at weblogic/servlet/internal/FilterChainImpl.doFilter(FilterChainImpl.java:42(Compiled Code)) at weblogic/servlet/internal/WebAppServletContext$ServletInvocationAction.run(WebAppServletContext.java:3486(Compiled Code)) at weblogic/security/acl/internal/AuthenticatedSubject.doAs(AuthenticatedSubject.java:321(Compiled Code)) at weblogic/security/service/SecurityManager.runAs(Bytecode PC:18(Compiled Code)) at weblogic/servlet/internal/WebAppServletContext.securedExecute(WebAppServletContext.java:2120(Compiled Code)) at weblogic/servlet/internal/WebAppServletContext.execute(WebAppServletContext.java:2086(Compiled Code)) at weblogic/servlet/internal/ServletRequestImpl.run(ServletRequestImpl.java:1406(Compiled Code)) at weblogic/work/ExecuteThread.execute(ExecuteThread.java:201(Compiled Code)) at weblogic/work/ExecuteThread.run(ExecuteThread.java:173(Compiled Code)) Native Stack (0xD2647A48 [libj9prt24.so+0x9a48]) (0xD2BA2C38 [libj9dmp24.so+0x10c38]) (0xD263FBAC [libj9prt24.so+0x1bac]) (0xD2BA0AB4 [libj9dmp24.so+0xeab4]) (0xD2B9F0CC [libj9dmp24.so+0xd0cc]) (0xD263FBAC [libj9prt24.so+0x1bac]) (0xD2B9ED10 [libj9dmp24.so+0xcd10]) (0xD2BA5280 [libj9dmp24.so+0x13280]) (0xD2B944DC [libj9dmp24.so+0x24dc]) (0xD2B983A0 [libj9dmp24.so+0x63a0]) (0xD263FBAC [libj9prt24.so+0x1bac]) (0xD2B98344 [libj9dmp24.so+0x6344]) (0xD2B98130 [libj9dmp24.so+0x6130]) (0xD2BB3A94 [libj9dmp24.so+0x21a94]) (0xD2BB3ED0 [libj9dmp24.so+0x21ed0]) (0xD0ED2204 [libj9hookable24.so+0x204]) (0xD4007228 [libj9vm24.so+0x9228]) (0xD4007C8C [libj9vm24.so+0x9c8c]) (0xD5016A8C [libj9jit24.so+0x56aa8c]) (0xD4009910 [libj9vm24.so+0xb910]) (0xD263FBAC [libj9prt24.so+0x1bac]) (0xD4009830 [libj9vm24.so+0xb830]) (0xD2197CC0 [libj9thr24.so+0x1cc0]) _pthread_body+0xec (0xD04EFC50 [libpthreads.a+0x3c50]) Number of loaded classes in Java heap : 34,648 Number of classloaders in Java heap : 9,321 Recommended -Xmxcl setting (only for IBM Java 5.0, up to and including Service Refresh 4 (build date:February 1st ,2007)) : 12,117 or greater note: oNLY FOR jAVA 5.0 sERVICE rEFRESH 4 (BUILD DATE:fEBRUARY 1ST, 2007) AND OLDER. wHEN YOU USE DELEGATED CLASS LOADERS, THE jvm CAN CREATE A LARGE NUMBER OF cLASSlOADER OBJECTS. oN ibm jAVA 5.0 sERVICE rEFRESH 4 AND OLDER, THE NUMBER OF CLASS LOADERS THAT ARE PERMITTED IS LIMITED TO 8192 BY DEFAULT AND AN oUToFmEMORYeRROR EXCEPTION IS THROWN WHEN THIS LIMIT IS EXCEEDED. uSE THE -xMXCL PARAMETER TO INCREASE THE NUMBER OF CLASS LOADERS ALLOWED TO AVOID THIS PROBLEM, FOR EXAMPLE TO 25000, BY SETTING -xMXCL25000, UNTIL THE PROBLEM IS RESOLVED. pLEASE EXAMINE THE CURRENT THREAD STACK TRACE TO CHECK WHETHER A CLASS LOADER IS BEING LOADED IF THERE IS AN oUToFmEMORYeRROR. fOR EXAMPLE, THE FOLLOWING STACK TRACE INDICATES THAT A CLASS LOADER IS BEING LOADED: AT COM/IBM/OTI/VM/vm.INITIALIZEcLASSlOADER(nATIVE mETHOD) AT JAVA/LANG/cLASSlOADER. (cLASSlOADER.JAVA:120) cOMMAND LINE : /USR/JAVA6/BIN/JAVA -xMS512M -xMX1024M -DA -dPLATFORM.HOME=/HOME/EBILLS/BEA/WLSERVER_10.3 -dWLS.HOME=/HOME/EBILLS/BEA/WLSERVER_10.3/SERVER -dWEBLOGIC.HOME=/HOME/EBILLS/BEA/WLSERVER_10.3/SERVER -dWEBLOGIC.MANAGEMENT.DISCOVER=TRUE -dWLW.ITERATIVEdEV=FALSE -dWLW.TESTcONSOLE=FALSE -dWLW.LOGeRRORStOcONSOLE= -dCLIENT.ENCODING.OVERRIDE=gbk -dFILE.ENCODING=gbk -dUSER.LANGUAGE=ZH -dUSER.REGION=cn -dDEFAULT.CLIENT.ENCODING=gbk -dWEBLOGIC.THREADPOOL.mINpOOLsIZE=200 -dWEBLOGIC.THREADPOOL.mAXpOOLsIZE=500 -dJAVA.AWT.HEADLESS=TRUE -dWEBLOGIC.EXT.DIRS=/HOME/EBILLS/BEA/PATCH_WLS1030/PROFILES/DEFAULT/SYSEXT_MANIFEST_CLASSPATH:/HOME/EBILLS/BEA/PATCH_CIE660/PROFILES/DEFAULT/SYSEXT_MANIFEST_CLASSPATH -dWEBLOGIC.MANAGEMENT.USERNAME=WEBLOGIC -dWEBLOGIC.MANAGEMENT.PASSWORD=WEBLOGIC -dWEBLOGIC.nAME=aDMINsERVER -dJAVA.SECURITY.POLICY=/HOME/EBILLS/BEA/WLSERVER_10.3/SERVER/LIB/WEBLOGIC.POLICY WEBLOGIC.sERVER tHREAD sTATUS aNALYSIS sTATUS nUMBER OF tHREADS : 287 Percentage Deadlock 0 0 (%) Runnable 6 2 (%) Waiting on condition 263 92 (%) Waiting on monitor 0 0 (%) Suspended 0 0 (%) Object.wait() 0 0 (%) Blocked 16 6 (%) Parked 2 1 (%) Thread Method Analysis Method Name Number of Threads : 287 Percentage java/lang/Object.wait(Native Method) 167 58 (%) java/lang/Thread.sleep(Native Method) 22 8 (%) NO JAVA STACK 13 5 (%) weblogic/socket/PosixSocketMuxer.processSockets(PosixSocketMuxer.java:93(Compiled Code)) 10 3 (%) weblogic/timers/internal/TimerManagerImpl.complete(TimerManagerImpl.java:654(Compiled Code)) 6 2 (%) java/lang/Throwable.fillInStackTrace(Native Method) 5 2 (%) java/net/PlainSocketImpl.socketAccept(Native Method) 5 2 (%) weblogic/servlet/internal/session/SessionContext$SessionInvalidator.timerExpired(SessionContext.java:788(Compiled Code)) 2 1 (%) sun/misc/Unsafe.park(Native Method) 2 1 (%) java/util/AbstractList.iterator(AbstractList.java:583(Compiled Code)) 2 1 (%) java/io/ObjectInputStream$BlockDataInputStream.readUTFBody(ObjectInputStream.java:3036(Compiled Code)) 2 1 (%) java/lang/StringBuilder.toString(StringBuilder.java:803(Compiled Code)) 2 1 (%) sun/reflect/UTF8.encode(UTF8.java:33(Compiled Code)) 2 1 (%) java/nio/CharBuffer.wrap(CharBuffer.java:361(Compiled Code)) 2 1 (%) java/util/Hashtable.newEntry(Hashtable.java:91(Compiled Code)) 2 1 (%) java/util/regex/Pattern.newSingle(Pattern.java:2962(Compiled Code)) 1 0 (%) java/util/Vector. (Vector.java:76(Compiled Code)) 1 0 (%) java/util/TreeMap.putImpl(TreeMap.java:4544(Compiled Code)) 1 0 (%) java/util/Hashtable.put(Hashtable.java:769(Compiled Code)) 1 0 (%) java/util/Hashtable.clone(Hashtable.java:327(Compiled Code)) 1 0 (%) java/util/AbstractList$FullListIterator. (AbstractList.java:94(Compiled Code)) 1 0 (%) java/util/AbstractCollection.toArray(AbstractCollection.java:352(Compiled Code)) 1 0 (%) java/security/SecureRandom.nextBytes(SecureRandom.java:292(Compiled Code)) 1 0 (%) java/net/SocketInputStream.read(SocketInputStream.java:179(Compiled Code)) 1 0 (%) java/net/Inet6AddressImpl.lookupAllHostAddr(Native Method) 1 0 (%) java/lang/reflect/Method.invoke(Method.java:611(Compiled Code)) 1 0 (%) java/lang/reflect/Array.newArrayImpl(Native Method) 1 0 (%) java/lang/Throwable.printStackTrace(Throwable.java:363(Compiled Code)) 1 0 (%) java/lang/Throwable.printStackTrace(Throwable.java:338(Compiled Code)) 1 0 (%) java/lang/StringBuilder.ensureCapacityImpl(StringBuilder.java:339(Compiled Code)) 1 0 (%) java/lang/String.toUpperCase(String.java:1300(Compiled Code)) 1 0 (%) java/lang/String.toUpperCase(String.java:1129(Compiled Code)) 1 0 (%) java/lang/String. (String.java:350(Compiled Code)) 1 0 (%) java/io/ObjectStreamClass.lookup(ObjectStreamClass.java:291(Compiled Code)) 1 0 (%) java/io/ObjectStreamClass.lookup(ObjectStreamClass.java:287(Compiled Code)) 1 0 (%) java/io/ObjectInputStream.readString(ObjectInputStream.java:1629(Compiled Code)) 1 0 (%) com/ibm/tools/attach/javaSE/IPC.waitSemaphore(Native Method) 1 0 (%) com/ibm/misc/SignalDispatcher.waitForSignal(Native Method) 1 0 (%) com/ibm/lang/management/OperatingSystemNotificationThread.processNotificationLoop(Native Method) 1 0 (%) com/ibm/lang/management/MemoryNotificationThread.processNotificationLoop(Native Method) 1 0 (%) com/bea/security/xacml/combinator/standard/StandardRuleCombinerLibrary$3$1.evaluate(StandardRuleCombinerLibrary.java:200(Compiled Code)) 1 0 (%) com/amerisia/ebills/schedule/data/FileSource.findAll(FileSource.java:213(Compiled Code)) 1 0 (%) com/amerisia/ebills/commons/util/BaseDAO.loadAllRowBySql(BaseDAO.java:644(Compiled Code)) 1 0 (%) weblogic/utils/io/ChunkedDataOutputStream.makeChunkedDataInputStream(ChunkedDataOutputStream.java:376(Compiled Code)) 1 0 (%) weblogic/utils/http/QueryParams.getCurrent(QueryParams.java:21(Compiled Code)) 1 0 (%) weblogic/utils/collections/SecondChanceCacheMap.put(SecondChanceCacheMap.java:72(Compiled Code)) 1 0 (%) weblogic/socket/PosixSocketMuxer.poll(Native Method) 1 0 (%) weblogic/servlet/internal/MuxableSocketHTTP. (MuxableSocketHTTP.java:112(Compiled Code)) 1 0 (%) weblogic/security/service/SecurityManager.runAs(Bytecode PC:18(Compiled Code)) 1 0 (%) weblogic/security/service/JNDIResource.initialize(JNDIResource.java:146(Compiled Code)) 1 0 (%) weblogic/jndi/internal/BasicNamingNode.getPrefix(BasicNamingNode.java:984(Compiled Code)) 1 0 (%) sun/util/calendar/ZoneInfo.getOffsets(ZoneInfo.java:237(Compiled Code)) 1 0 (%) sun/reflect/GeneratedMethodAccessor120.invoke(Bytecode PC:0(Compiled Code)) 1 0 (%) org/apache/struts/util/RequestUtils.populate(RequestUtils.java:399(Compiled Code)) 1 0 (%) oracle/net/ano/AnoComm.o(Bytecode PC:1(Compiled Code)) 1 0 (%) oracle/jdbc/driver/T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:801(Compiled Code)) 1 0 (%) oracle/jdbc/driver/T4CMAREngine.unmarshalKEYVAL(T4CMAREngine.java:1859(Compiled Code)) 1 0 (%) oracle/jdbc/driver/DBConversion.stringToDriverCharBytes(DBConversion.java:443(Compiled Code)) 1 0 (%) Thread Aggregation Analysis Thread Type Number of Threads : 287 Percentage Thread 276 96 (%)
©️2020 CSDN 皮肤主题: 设计师:Defonds 返回首页