Export fails or AnalysisUI reports no datasource found

Technote (troubleshooting)


Problem(Abstract)

Data Export fails at the end of index build step or AnalysisUI reports no datasource avaailable in IBM Social Media Analytics (SMA) after total memory being increased or after applying SUSE Fixpack.

Symptom

At one of the following two symptons may be observed:

1..Overall status on Configuration UI shows export failure.

The following similar entries could be found in DASServlet.<project id>.0.log on data node under /local/cci/prod/dls/data/indexoutput/<project id>/dasSources/logs

    SEVERE: Failed to init ILEL source!
    Throwable occurred: java.io.IOException: Map failed
    at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:772)
    at org.apache.lucene.store.MMapDirectory$MMapIndexInput.<init>(MMapDirectory.java:264)
    at org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:216)
    at org.apache.lucene.index.FieldsReader.<init>(FieldsReader.java:129)
    at org.apache.lucene.index.SegmentCoreReaders.openDocStores(SegmentCoreReaders.java:244)
    at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:116)
    at org.apache.lucene.index.SegmentReader.get(SegmentReader.java:92)
    at org.apache.lucene.index.DirectoryReader.<init>(DirectoryReader.java:113)
    at org.apache.lucene.index.ReadOnlyDirectoryReader.<init>(ReadOnlyDirectoryReader.java:29)

    Further in same log file, an error of "OutOfMemory: Map failed" may be found as well:
    Caused by: java.lang.OutOfMemoryError: Map failed
    at sun.nio.ch.FileChannelImpl.map(FileChannelImpl.java:769)
    ... 37 more
2. After a successful end to end job run, AnalysisUI still reports no datasource found and prompts to run a complete end to end job.

In both cases, setting excessively high java heap to DAS JVM will not resolve the problem at all.

Cause

This error is not indication of memory issues within DAS JVM heap. Lucene index building is using memory mapped file using nio (sun.nio.ch.FileChannelImpl.map ), which uses memory outside of preset JAVA heap. The default SLES 11 FP2 installation allocates fixed amount of virtual memory for OS to use.

Resolving the problem

1. Logon to data node as cciusr user

2. Run ulimit -a

    ..
    max memory size (kbytes, -m) 13997628
    max user processes (-u) 256822
    virtual memory (kbytes, -v) 27990240
    file locks (-x) unlimited
    ...
3. Check the entry of virtual memory and max memory size, both values should be set to unlimited
4. Add the following entry to the end of /home/cciusr/.bashrc, or /etc/bash.bashrc (requires root permission)
    ulimit -v unlimited
    ulimit -m unlimited
5. Open a new terminal window and restart SMA services.
6. Export again after Social Media Analytics is running.

Rate this page:

(0 users)Average rating

Document information


More support for:

Social Media Analytics

Software version:

1.2, 1.3

Operating system(s):

Linux

Software edition:

All Editions

Reference #:

1634503

Modified date:

2014-04-15

Translate my page

Machine Translation

Content navigation