ESGF_Best_Practices

ESGF Node Administrator Best Practices

This document collects some of the recommended best practices for configuring and operating an ESGF Node.

Tomcat

• Run Tomcat with extended memory, if available. Tomcat settings can be configured by inserting the following lines in the file /etc/esg.env:

  • export JAVA_OPTS='-Xmx2048m -Xms1024m -XX:MaxPermSize=1024m'

• Run Tomcat with SSL debug turned on, but only when trying to solve a security issue, otherwise the output is too verbose. In file /etc/esg.env:

  • export CATALINA_OPTS=-Djavax.net.debug=ssl

Solr

• Run Jetty (the Solr container) with extended memory, if available. For example, insert the following line in the file /etc/esg.env:

  • export SOLR_JAVA_OPTS_='-Xmx1024m -Xms512m -XX:MaxPermSize=512m'
    (these settings will be applied upon restart to the local master and slave Solr, as well as to all deployed replica shards)

• Optimizing the Solr index (i.e. reducing the number of its fragments) will boost the search performance at the local Node.

  • For one-time optimization, use the esg-node script with the --optimize-index option, which will optimize all the configured shards' cores.

    • %> esg-node --optimize-index

  • Better yet, run a optimization script every night to optimize each Solr instance (all cores for that instance). An example script for the master Solr is provided below.

    • curl -s http://localhost:8984/solr/datasets/update?commit=true -H "Content-Type:text/xml" --data-binary ''
    • curl -s http://localhost:8984/solr/files/update?commit=true -H "Content-Type:text/xml" --data-binary ''
    • curl -s http://localhost:8984/solr/aggregations/update?commit=true -H "Content-Type:text/xml" --data-binary ''