Help:Toolforge/Elasticsearch: Difference between revisions

Content deleted Content added

Inline

Revision as of 22:00, 4 November 2016

Elasticsearch is a full text search system built on Apache Lucene. It can be used to index and search data stored as JSON documents. It is the technology used to power Wikimedia's CirrusSearch system.

An Elasticsearch cluster that can be used by all tools is available on tools-elastic-0[123], on the non-standard port 80. This Elasticsearch cluster is a shared resource and all documents indexed in it can be read by anonymous users from within Tool Labs. Write access needed to create new indexes, and store or update documents requires a username and password.

Read-only access

The Elasticsearch servers allow anyone to read any of the indexes that it contains. This access is limited to other hosts in the Tool Labs project (e.g. the OGE job grid, Kubernetes containers, and the bastion servers).

The Elasticsearch service is available on port 80 on the following servers:

Note that the default Elasticsearch port (9200) is not used.

Write access

Elasticsearch does not offer multi-tenant access control in their open source version, so write access to the indices is only loosely protected. PUT, POST, or DELETE requests sent to the Elasticsearch servers require HTTP Basic Authentication using a username and password specific to each tool.

Requests for write access are made by filing a Phabricator task. When credentials have been created they will be placed in /data/project/$TOOL/.elasticsearch.ini. Access requests are currently processed manually and may take a few days to be fulfilled.

@@ Line 1: / Line 1: @@
 <noinclude>{{Template:Tool Labs nav}}</noinclude>
+'''[https://www.elastic.co/products/elasticsearch Elasticsearch]''' is a full text search system built on Apache Lucene. It can be used to index and search data stored as JSON documents. It is the technology used to power Wikimedia's CirrusSearch system.
-Tool Labs provides a shared Elasticsearch cluster for use by tools and bots needing modern full text search capabilities.
+An Elasticsearch cluster that can be used by all tools is available on <code>tools-elastic-0[123]</code>, on the non-standard port <code>80</code>. This Elasticsearch cluster is a shared resource and all documents indexed in it can be read by anonymous users from within Tool Labs. Write access needed to create new indexes, and store or update documents requires a username and password.
+<includeonly>See full documentation at [[/Elasticsearch]] for more information.</includeonly><noinclude>
 == Read-only access ==
 The Elasticsearch servers allow anyone to read any of the indexes that it contains. This access is limited to other hosts in the Tool Labs project (e.g. the OGE job grid, Kubernetes containers, and the bastion servers).
@@ Line 18: / Line 21: @@
 Requests for write access are made by [https://phabricator.wikimedia.org/maniphest/task/create/?projects=Tool-Labs&title=Elasticsearch%20credential%20request%20for%20YOUR_TOOL_NAME_HERE&description=DESCRIBE%20USE%20OF%20ELASTICSEARCH%20HERE&subscribers=bd808 filing a Phabricator task]. When credentials have been created they will be placed in <code>/data/project/$TOOL/.elasticsearch.ini</code>. Access requests are currently processed manually and may take a few days to be fulfilled.
+</noinclude>