EPollArrayWrapper.epollWait 100% CPU Usage

Hi,

I believe I have an issue similar to #302 but on Linux (Ubuntu 10.04) with JDK (1.6.0u30) and JDK(1.7.0u4) using Netty-4.0.0 (Revision: 52a7d28cb59e3806fda322aecf7a85a6adaeb305)

The app is proxying connections to backend systems.  The proxy has a pool of channels that it can use to send requests to the backend systems.  If the pool is low on channels, new channels are spawned and put into the pool so that requests sent to the proxy can be serviced.  The pools get populated on app startup, so that is why it doesn't take long at all for the CPU to spike through the roof (22 seconds into the app lifecycle).

The test box has two CPUs, the output from 'top' is below:

```
PID  USER   PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
8220 root   20   0 2281m 741m  10m R 50.2 18.7  0:22.57 java                                                                             
8218 root   20   0 2281m 741m  10m R 49.9 18.7  0:22.65 java                                                                             
8219 root   20   0 2281m 741m  10m R 49.2 18.7  0:22.86 java                                                                             
8221 root   20   0 2281m 741m  10m R 49.2 18.7  0:22.20 java 
```

Thread Dump for the four NioClient based Worker Threads that are chewing up all the CPU.

```
"backend-worker-pool-7-thread-1" prio=10 tid=0x00007f5918015800 nid=0x201a runnable [0x00007f5924ba3000]
   java.lang.Thread.State: RUNNABLE
    at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
    at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
    at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
    at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
    - locked <0x000000008be93580> (a sun.nio.ch.Util$2)
    - locked <0x000000008be93570> (a java.util.Collections$UnmodifiableSet)
    - locked <0x000000008be92548> (a sun.nio.ch.EPollSelectorImpl)
    at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
    at io.netty.channel.socket.nio.SelectorUtil.select(SelectorUtil.java:55)
    at io.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:261)
    at io.netty.channel.socket.nio.NioWorker.run(NioWorker.java:37)
    at io.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:43)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:662)   Locked ownable synchronizers:    - <0x000000008be00748> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)
```

```
"backend-worker-pool-7-thread-2" prio=10 tid=0x00007f5918012000 nid=0x201b runnable [0x00007f5924b82000]
   java.lang.Thread.State: RUNNABLE
    at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
    at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
    at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
    at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
    - locked <0x000000008be94a28> (a sun.nio.ch.Util$2)
    - locked <0x000000008be94a18> (a java.util.Collections$UnmodifiableSet)
    - locked <0x000000008be90648> (a sun.nio.ch.EPollSelectorImpl)
    at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
    at io.netty.channel.socket.nio.SelectorUtil.select(SelectorUtil.java:55)
    at io.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:261)
    at io.netty.channel.socket.nio.NioWorker.run(NioWorker.java:37)
    at io.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:43)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:662)   Locked ownable synchronizers:    - <0x000000008be904c8> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)
```

```
"backend-worker-pool-7-thread-3" prio=10 tid=0x00007f5918007800 nid=0x201c runnable [0x00007f5924b61000]
   java.lang.Thread.State: RUNNABLE
    at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
    at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
    at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
    at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
    - locked <0x000000008be952e0> (a sun.nio.ch.Util$2)
    - locked <0x000000008be952d0> (a java.util.Collections$UnmodifiableSet)
    - locked <0x000000008be8f858> (a sun.nio.ch.EPollSelectorImpl)
    at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
    at io.netty.channel.socket.nio.SelectorUtil.select(SelectorUtil.java:55)
    at io.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:261)
    at io.netty.channel.socket.nio.NioWorker.run(NioWorker.java:37)
    at io.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:43)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:662)   Locked ownable synchronizers:    - <0x000000008be8f618> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)
```

```
"backend-worker-pool-7-thread-4" prio=10 tid=0x00007f5918019000 nid=0x201d runnable [0x00007f5924b40000]
   java.lang.Thread.State: RUNNABLE
    at sun.nio.ch.EPollArrayWrapper.epollWait(Native Method)
    at sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:210)
    at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:65)
    at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
    - locked <0x000000008be003f8> (a sun.nio.ch.Util$2)
    - locked <0x000000008be003e8> (a java.util.Collections$UnmodifiableSet)
    - locked <0x000000008be00408> (a sun.nio.ch.EPollSelectorImpl)
    at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
    at io.netty.channel.socket.nio.SelectorUtil.select(SelectorUtil.java:55)
    at io.netty.channel.socket.nio.AbstractNioWorker.run(AbstractNioWorker.java:261)
    at io.netty.channel.socket.nio.NioWorker.run(NioWorker.java:37)
    at io.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:43)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
    at java.lang.Thread.run(Thread.java:662)   Locked ownable synchronizers:    - <0x000000008be004e0> (a java.util.concurrent.locks.ReentrantLock$NonfairSync)
```


Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

EPollArrayWrapper.epollWait 100% CPU Usage #327

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Uh oh!

EPollArrayWrapper.epollWait 100% CPU Usage #327

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions