Help:CirrusSearch elasticsearch replicas: Difference between revisions
Ebernhardson (talk | contribs) →Accessing: use lvs dns instead of exact servers |
Ebernhardson (talk | contribs) No edit summary |
||
| Line 29: | Line 29: | ||
The set of indices that exist in a cluster can be queried through the elasticsearch [https://www.elastic.co/guide/en/elasticsearch/reference/6.5/cat-indices.html cat indices] API. |
The set of indices that exist in a cluster can be queried through the elasticsearch [https://www.elastic.co/guide/en/elasticsearch/reference/6.5/cat-indices.html cat indices] API. |
||
<code>curl -XGET http<nowiki>s</nowiki>:// |
<code>curl -XGET http<nowiki>s</nowiki>://cloudelastic.wikimedia.org:9843/_cat/indices</code> |
||
=== Schema === |
=== Schema === |
||
| Line 39: | Line 39: | ||
==== Query all indices ==== |
==== Query all indices ==== |
||
<code>curl -XGET http<nowiki>s</nowiki>:// |
<code>curl -XGET http<nowiki>s</nowiki>://cloudelastic.wikimedia.org:8243/*,*:*/_search?q=example</code> |
||
==== Query all content indices ==== |
==== Query all content indices ==== |
||
<code>curl -XGET http<nowiki>s</nowiki>:// |
<code>curl -XGET http<nowiki>s</nowiki>://cloudelastic.wikimedia.org:8243/*_content,*:*_content/_search?q=example</code> |
||
==== Fetch full document for single page by page id ==== |
==== Fetch full document for single page by page id ==== |
||
<code>curl -XGET http<nowiki>s</nowiki>:// |
<code>curl -XGET http<nowiki>s</nowiki>://cloudelastic.wikimedia.org:8243/enwiki_content/page/33179123</code> |
||
==== Fetch full document for single page by title ==== |
==== Fetch full document for single page by title ==== |
||
<code>curl -XGET http<nowiki>s</nowiki>:// |
<code>curl -XGET http<nowiki>s</nowiki>://cloudelastic.wikimedia.org:8243/enwiki_content/_search?q=title.keyword:Elasticsearch</code> |
||
Revision as of 17:02, 3 September 2019
Cloud Elastic is a replica of the CirrusSearch elasticsearch indices made available to Wikimedia Cloud Services applications (both Cloud VPS and Toolforge). These servers are not generally accessible from the internet at large, rather they are only accessible through applications running inside Cloud Services. Applications can use the full power of the elasticsearch search API's to query the search indices in ways that CirrusSearch does not expose directly on the wiki's themselves.
Accessing
There are actually three clusters, named chi, psi and omega. chi contains approximately the 200 largest wikis. psi and omega contain equal splits of the remaining smaller wikis.
| Cluster Name | URL |
|---|---|
| chi | https://cloudelastic.wikimedia.org:8243/
|
| psi | https://cloudelastic.wikimedia.org:8643/
|
| omega | https://cloudelastic.wikimedia.org:8443/
|
Clusters can be accessed through each other using the elasticsearch cross cluster search syntax. For example labswiki (wikitech's internal database name), which lives on the omega cluster, can be queried through the chi cluster with:
curl -XGET https://cloudelastic.wikimedia.org:8243/omega:labswiki/_search?q=example
Indices Available
All wikis have two indices, of the format <dbname>_content and <dbname>_general. The content index contains all of the content namespaces of the wiki, the general index contains everything else. So for example on wikipedia's articles are found in the content index, and talk pages are found in the general index. Querying both indices can be done through an alias by providing only the wiki db name.
The set of indices that exist in a cluster can be queried through the elasticsearch cat indices API.
curl -XGET https://cloudelastic.wikimedia.org:9843/_cat/indices
Schema
See mw:Extension:CirrusSearch/Schema.
Example Use Cases
Query all indices
curl -XGET https://cloudelastic.wikimedia.org:8243/*,*:*/_search?q=example
Query all content indices
curl -XGET https://cloudelastic.wikimedia.org:8243/*_content,*:*_content/_search?q=example
Fetch full document for single page by page id
curl -XGET https://cloudelastic.wikimedia.org:8243/enwiki_content/page/33179123
Fetch full document for single page by title
curl -XGET https://cloudelastic.wikimedia.org:8243/enwiki_content/_search?q=title.keyword:Elasticsearch