Why specify fields by type instead of index in Elasticsearch?


Eric Anastas

If multiple types in an Elasticsearch index have fields with the same name, those fields must have the same mapping, which tries to create the "foobar" property (string and long).

For example, if you try to put the following index mapping:

{
  "mappings": {
    "type_one": {
      "properties": {
        "foobar": { 
          "type": "string"
        }
      }
    },
    "type_two": {
      "properties": {
        "foobar": { 
          "type": "long"
        }
      }
    }
  }
}

...will return the following error

{
  "error": {
    "root_cause": [
      {
        "type": "mapper_parsing_exception",
        "reason": "Failed to parse mapping [type_one]: mapper [foobar] cannot be changed from type [long] to [string]"
      }
    ],
    "type": "mapper_parsing_exception",
    "reason": "Failed to parse mapping [type_one]: mapper [foobar] cannot be changed from type [long] to [string]",
    "caused_by": {
      "type": "illegal_argument_exception",
      "reason": "mapper [foobar] cannot be changed from type [long] to [string]"
    }
  },
  "status": 400
}

Here's what's from the elasticsearch website:

Conflicts between fields of different types

Fields with the same name in two different types with the same name must have the same mapping because they are internally backed by the same fields. Attempting to update a map parameter for a field where more than one type exists will throw an exception unless you specify the update_all_types parameter, in which case it will update the parameter in all fields with the same name in the same index.

If fields with the same name must have the same mapping for all types in the index, then why specify a field mapping for each type? Why not specify the fields for the entire index and then decide which fields to assign to each type.

For example something like this:

{
   "fields":{
      "PropA":{
         "type":"string"
      },
      "PropB":{
         "type":"long"
      },
      "PropC":{
         "type":"boolean"
      }
   },
   "types":{
      "foo":[
         "PropA",
         "PropB"
      ],
      "foo":[
         "PropA",
         "PropC"
      ],
      "foo":[
         "PropA",
         "PropC",
         "PropC"
      ]
   }
}

Wouldn't such a mapping format be more concise and better represent what is actually allowed?

The reason I'm asking is because I'm creating an index template JSON file that contains about 80 different fields used in 15 types. Many fields are available for multiple types, if not all types. So whenever I need to update a field, I have to make sure it is updated for every type that uses the field.

Eric Anastas

Looks like I'm not the only one confused.

Remove support for types? #15613

It sounds like removing support for multiple types per index and specifying fields at the index level is on the roadmap for a future release .

Related


Why do C++ bit fields require me to specify the type?

PhD Why do I have to specify a type for a bit field? struct MyBitField { unsigned int i : 4; } struct MyBitField2 { unsigned char i : 4; } What is the difference between the two? Why do I need to fully specify the type? asteroid with wings Bitfields

Why do C++ bit fields require me to specify the type?

PhD Why do I have to specify a type for a bit field? struct MyBitField { unsigned int i : 4; } struct MyBitField2 { unsigned char i : 4; } What is the difference between the two? Why do I need to fully specify the type? asteroid with wings Bitfields

Which fields in ElasticSearch index?

Taylor I ran a conversion program catmandu and sent the results to Elasticsearch . I am new to Elasticsearch. Do you know how to find something like ".schema" for sqlite3 in Elasticsearch? I would like to know what fields are there. After searching this mornin

Which fields in ElasticSearch index?

Taylor I ran a conversion program catmandu and sent the results to Elasticsearch . I am new to Elasticsearch. Do you know how to find something like ".schema" for sqlite3 in Elasticsearch? I would like to know what fields are there. After searching this mornin

Elasticsearch specify index filter in aggregation

Pramod Kumar I have an elastic query aggregation and I need to filter the aggregation based on the index name. The query part actually handles multiple indexes, but I want to filter aggregates for a specific index. Please help me how to pass index filter in ag

Elasticsearch specify index filter in aggregation

Pramod Kumar I have an elastic query aggregation and I need to filter the aggregation based on the index name. The query part actually handles multiple indexes, but I want to filter aggregates for a specific index. Please help me how to pass index filter in ag

Why specify a pointer type?

Jonis What actually happens when a pointer of a specific type is declared? Is it possible to specify types for pointers other than pointer arithmetic or indexing? Shravan40 The type of pointer is required in the following cases dereference pointer pointer arit

Why specify a pointer type?

Jonis What actually happens when a pointer of a specific type is declared? Is it possible to specify types for pointers other than pointer arithmetic or indexing? Shravan40 The type of pointer is required in the following cases dereference pointer pointer arit

Why specify a pointer type?

Jonis What actually happens when a pointer of a specific type is declared? Is it possible to specify types for pointers other than pointer arithmetic or indexing? Shravan40 The type of pointer is required in the following cases dereference pointer pointer arit

Why specify a pointer type?

Jonis What actually happens when a pointer of a specific type is declared? Is it possible to specify types for pointers other than pointer arithmetic or indexing? Shravan40 The type of pointer is required in the following cases dereference pointer pointer arit

Why specify a pointer type?

Jonis What actually happens when a pointer of a specific type is declared? Is it possible to specify types for pointers other than pointer arithmetic or indexing? Shravan40 The type of pointer is required in the following cases dereference pointer pointer arit

Can I specify result fields in elasticsearch query?

Shaheed In my dataset, documents contain more than 20 fields with nested objects. Most of them are long text fields. These fields are very important for full text search, but we only need the title, short description and ID in the output. Is it possible to spe

Specify fields using the Elasticsearch Transport client

Tula Is there a way to specify the fields that ES returns via the transport client (specifically using the BoolQueryBuilder)? Using the REST API, this seems easy, for example, can I specify the result field in the elasticsearch query? But not sure how to use t

Specify fields using the Elasticsearch Transport client

Tula Is there a way to specify the fields that ES returns via the transport client (specifically using the BoolQueryBuilder)? Using the REST API, this seems easy, for example, can I specify the result field in the elasticsearch query? But not sure how to use t

Fields found by index instead of name

Alessandro Cifani Here is my problem, I have this query: SQL = "SELECT * FROM Items, Genre, Artist WHERE Artist.ID = " & SelArtist & " AND Items.Artist = Artist.ID AND Items.Genre = Genre.ID" I need help from this

Fields found by index instead of name

Alessandro Cifani Here is my problem, I have this query: SQL = "SELECT * FROM Items, Genre, Artist WHERE Artist.ID = " & SelArtist & " AND Items.Artist = Artist.ID AND Items.Genre = Genre.ID" I need help from this

List all fields in elasticsearch index?

and How can I get a list of all fields present in the index (ie, the fields that appear in the indexed documents, not just in the map)? Bacchus Since 1.3 you have the _field_names meta field. { "aggs": { "Field names": { "terms": { "field":

Disable index encrypted fields in Elasticsearch

username I have an encrypted field stored in an elastic search index, but don't want to search through it, I just want to store this field in the document and return it when queried on other fields. Can I disable search for just this one string field? Motivati

Elasticsearch - How to index calculated fields?

Mela Is it possible to use a 'today' comparison for calculations in an index so that I can search and filter on calculated fields? E.g: Name: John Last Name: Smith Full Name: First Name + Last Name = John Smith Birthday: 22.01.1992 Age: Today - Birthday = 28 o

Disable index encrypted fields in Elasticsearch

username I have an encrypted field stored in an elastic search index, but don't want to search through it, I just want to store this field in the document and return it when queried on other fields. Can I disable search for just this one string field? Motivati

Disable index encrypted fields in Elasticsearch

username I have an encrypted field stored in an elastic search index, but don't want to search through it, I just want to store this field in the document and return it when queried on other fields. Can I disable search for just this one string field? Motivati

Disable index encrypted fields in Elasticsearch

username I have an encrypted field stored in an elastic search index, but don't want to search through it, I just want to store this field in the document and return it when queried on other fields. Can I disable search for just this one string field? Motivati

Disable index encrypted fields in Elasticsearch

username I have an encrypted field stored in an elastic search index, but don't want to search through it, I just want to store this field in the document and return it when queried on other fields. Can I disable search for just this one string field? Motivati

How to specify index and date/time fields in Timelion?

John D I'm trying to use the Timelion app in Kibana and I can't find where to specify the index name and time fields. Is there a way to do it on the fly, or does it have to be done in a config file somewhere? If so, where is that file? Andre Stefan .es(index=y

How to specify index and date/time fields in Timelion?

John D I'm trying to use the Timelion app in Kibana and I can't find where to specify the index name and time fields. Is there a way to do it on the fly, or does it have to be done in a config file somewhere? If so, where is that file? Andre Stefan .es(index=y