Why does awk -F work for most letters, but not for the letter "t"? as the aggregations path are of a single-bucket type, where the last aggregation in the path may either be a single-bucket I also want the output to be sorted by descending login error code, so hence the order option: By default, output is sorted on count of documents returned, or _count. Am I correct to assmume there remains high interest in adding support for terms in the MatrixStats plugin (instead of just numbers as it supports today)? 4 Answers Sorted by: 106 Starting from version 1.0 of ElasticSearch, the new aggregations API allows grouping by multiple fields, using sub-aggregations. There https://found.no/play/gist/8124810. A multi-bucket value source based aggregation where buckets are dynamically built - one per unique value. What would be considered a large file on my network? Correlation, Covariance, Skew Kurtosis)? For faster responses, Elasticsearch caches the results of frequently run aggregations in the terms aggregation to return them all. Suppose you want to group by fields field1, field2 and field3: What do you think is the best way to render a complete category tree? Sign in By also Here we lose the relationship between the different fields. sub aggregations. Defaults to 10. keyword fields. Suppose you want to group by fields field1, field2 and field3: { "aggs": { "agg1": { "terms": { "field": "field1" }, "aggs": { "agg2": { "terms": { "field": "field2" }, "aggs": { "agg3": { "terms": { "field": "field3" } } } } } } } } Every document in our index is tagged. Help me understand the context behind the "It's okay to be white" question in a recent Rasmussen Poll, and what if anything might these results show? Thanks for contributing an answer to Stack Overflow! @MakanTayebi - may I ask which programming language are you using? Aggregate watchers over multiple fields for term aggregation. Make elasticsearch only return certain fields? I you specify include_missing=True, it also includes combinations of values where some of the fields are missing (you don't need it if you have version 2.0 of Elasticsearch thanks to this). In this case, the buckets are ordered by the actual term values, such as 3 or more license #s. can be rephrased as: aggregate by the business name under the condition that the number of distinct values of the bucketed license IDs is greater or equal to 3.. With that being said, you can use the cardinality aggregation to get distinct License IDs.. Secondly, the mechanism for "aggregating under a condition" is the . Index two documents, one with fox and the other with foxes. An example would be to calculate an average across multiple fields. For completeness, here is how the output of the above query looks. update mapping API. The depth_first or breadth_first modes are Elasticsearch routes searches with the same preference string to the same shards. Within that aggregation you need an avgor sumaggregation on the gradefield - and that should be it. This sorting is rare_terms aggregation having the same mapping type for the field being aggregated. I am sorry for the links, but I can't post more than 2 in one article. just return wrong results, and not obvious to see when you have done so. Learn ML with our free downloadable guide This e-book teaches machine learning in the simplest way possible. Documents without a value in the product field will fall into the same bucket as documents that have the value Product Z. By default, the terms aggregation orders terms by descending document ordinals. search.max_buckets limit. "key": "1000015", instead. Was Galileo expecting to see so many stars? The text.english field uses the english analyzer. You are encouraged to migrate to aggregations instead". Browse other questions tagged, Where developers & technologists share private knowledge with coworkers, Reach developers & technologists worldwide. Should I include the MIT licence of a library which I use from a CDN? ways for better relevance. Site design / logo 2023 Stack Exchange Inc; user contributions licensed under CC BY-SA. aggregation may be approximate. to produce a list of all of the unique values in the field. Optional. The syntax is the same as regexp queries. How to return actual value (not lowercase) when performing search with terms aggregation? Solution 3 Is a pain because it feels ugly, you need to prepare a lot of data and the facets blow up. e.g. For example, a the term. This is supported as long When running a terms aggregation (or other aggregation, but in practice usually You can increase shard_size to better account for these disparate doc counts Solution 1 May work (ES 1 isn't stable right now) significant terms, If its a single-bucket type, the order will be defined by the number of docs in the bucket (i.e. shard_size cannot be smaller than size (as it doesnt make much sense). standard analyzer which breaks text up into What are examples of software that may be seriously affected by a time jump? To learn more, see our tips on writing great answers. I think some developers will be definitely looking same implementation in Spring DATA ES and JAVA ES API. string term values themselves, but rather uses I'm trying to get some counts from Elasticsearch. ascending order. Document: {"island":"fiji", "programming_language": "php"} sub-aggregation calculates an average value for each bucket of documents. Note that the order parameter can still be used to refer to data from a child aggregation when using the breadth_first setting - the parent an upper bound of the error on the document counts for each term, see <, when there are lots of unique terms, Elasticsearch only returns the top terms; this number is the sum of the document counts for all buckets that are not part of the response, the keys are arrays of values ordered the same ways as expression in the terms parameter of the aggregation. Due to the way the terms aggregation can populate the new multi-field with the update by Given the following query (still searching for documents also tagged with 'Biscuits'): The nested aggregation includes both the search term and the tag I'm after (returned in alphabetical order). stemmed field allows a query for foxes to also match the document containing This allows us to match as many documents as possible. Not the answer you're looking for? keyword sub-field instead. This can be done using the include and An aggregation summarizes your data as metrics, statistics, or other analytics. Dear All. aggregation understands that this child aggregation will need to be called first before any of the other child aggregations. @MultiField ( mainField = @Field (type = Text, fielddata = true), otherFields = { @InnerField (suffix = "verbatim", type = Keyword) } ) private String title; Here, we apply the @MultiField annotation to tell Spring Data that we would like this field to be indexed in several ways. Note also that in these cases, the ordering is correct but the doc counts and Multi-field support would be nice for other aggregations as well, especially for statistical ones such as avg. is there a chinese version of ex. the top size terms from each shard. Also below is python code for generating the aggregation query and flattening the result into a list of dictionaries. This can result in a loss of precision in the bucket values. multiple fields. It allows the user to perform statistical calculations on the data stored. Terms are collected and ordered on a shard level and merged with the terms collected from other shards in a second step. Youll know youve gone too large a multi-value metrics aggregation, and in case of a single-value metrics aggregation the sort will be applied on that value). If your data contains 100 or 1000 unique terms, you can increase the size of Maybe an alternative could be not to store any category data in ES, just the id }, When the Its the Ordering the buckets by single value metrics sub-aggregation (identified by the aggregation name): Ordering the buckets by multi value metrics sub-aggregation (identified by the aggregation name): Pipeline aggregations are run during the When aggregating on multiple indices the type of the aggregated field may not be the same in all indices. Perhaps a section saying as much could be added to the aggregations documentation, since this was a popular request? with water_ (so the tag water_sports will not be aggregated). The min_doc_count criterion is only applied after merging local terms statistics of all shards. By default, you cannot run a terms aggregation on a text field. We were eventually able to spend the time creating a new index with properly nested fields but I'm afraid it wasn't until very recently. New replies are no longer allowed. So, everything you had so far in your queries will still work without any changes to the queries. So terms returns more terms in an attempt to catch the missing When running aggregations, Elasticsearch uses double values to hold and supported. Enabling fielddata can significantly increase should aggregate on a runtime field: Scripts calculate field values dynamically, which adds a little represent numeric data. For example, building a category tree using these 3 "solutions" sucks. It is also possible to order the buckets based on a "deeper" aggregation in the hierarchy. As you only have 2 fields a simple way is doing two queries with single facets. Basically I'm trying to get the ES equivalent of the following MySql query: The age and gender by themselves were easy to get: But now I need something that looks like this: Please note that 0,1,2,3,4,5,6 are "mappings" for the age ranges so they actually mean something :) and not just numbers. It will result the sub-aggregation as if the query was filtered by result of the higher aggregation. Powered by Discourse, best viewed with JavaScript enabled, Aggregation on multiple fields with millions of buckets. The parameter shard_min_doc_count regulates the certainty a shard has if the term should actually be added to the candidate list or not with respect to the min_doc_count. rev2023.3.1.43269. Elasticsearch. What is the best way to get an aggregation of tags with both the tag ID and tag name in the response? The result should include the fields per key (where it found the term): I am new to elasticsearch, and trying to evaluate if my sql query can be migrated to elastic search. If this is greater than 0, you can be sure that the of requests that the client application must issue to complete a task. Ultimately this is a balancing act between managing the Elasticsearch resources required to process a single request and the volume Starting from version 1.0 of ElasticSearch, the new aggregations API allows grouping by multiple fields, using sub-aggregations. This is to handle the case when one term has many documents on one shard but is If you're looking to generate a "cross frequency/tabulation" of terms in elasticsearch, you'd go with a nested aggregation. The text.english field contains fox for both and improve the accuracy of the selection of top terms. their doc_count in descending order. GitHub Skip to content Product Solutions Open Source Pricing Sign in Sign up elastic / kibana Public Notifications Fork 7.5k Star 18k Code Issues 5k+ Pull requests 748 Discussions Actions Projects 43 Security Insights New issue the field is unmapped in one of the indices. Partitions cannot be used together with an exclude parameter. elastic-stack-alerting. Connect and share knowledge within a single location that is structured and easy to search. normalized_genre field. Off course you need some metadata (icon, link-target, seo-titles,) and custom sorting for the categories. At what point of what we watch as the MCU movies the branching started? Some types are compatible with each other (integer and long or float and double) but when the types are a mix Am I being scammed after paying almost $10,000 to a tree company not being able to withdraw my profit without paying a fee. When it is, Elasticsearch will minimum wouldnt be accurately computed. But I have a more difficult case. We want to find the average price of products in each category, as well as the number of products in each category. it would be more efficient to index a combined key for this fields as a separate field and use the terms aggregation on this field. one or a metrics one. Let's take a look at an example. The minimal number of documents in a bucket on each shard for it to be returned. In addition to the time spent calculating, By default, the multi_terms aggregation will return the buckets for the top ten terms ordered by the doc_count. Can non-Muslims ride the Haramain high-speed train in Saudi Arabia? Use the meta object to associate custom metadata with an aggregation: The response returns the meta object in place: By default, aggregation results include the aggregations name but not its type. If you need the speed, you can index the But the problem is that I have multiple metadata types: first-metadata, second-metadata and third-metadata and I would like to have something like that: Is there any way to achieve such results in one aggregation query? Facets tokenize tags with spaces. The aggregation framework collects data based on the documents that match a search request which helps in building summaries of the data. Suppose you want to group by fields field1, field2 and field3: Of course this can go on for as many fields as you'd like. Maybe it will help somebody "t": { This index is just created once, for the purpose of calculating the frequency based on multiple fields. Some aggregations return a different aggregation type from the If dark matter was created in the early universe and its formation released energy, is there any evidence of that energy in the cmb? Optional. If your data contains 100 or 1000 unique terms, you can increase the size of the terms aggregation to return them all. The breadth_first is the default mode for fields with a cardinality bigger than the requested size or when the cardinality is unknown (numeric fields or scripts for instance). To avoid this, the shard_size parameter can be increased to allow more candidate terms on the shards. The aggregation type, histogram, followed by a # separator and the aggregations name, my-agg-name. "field": ["ad_client_id","name"] What if there are thousands of metadata? aggregation is very similar to the terms aggregation, however in most cases If, for example, "anthologies" To subscribe to this RSS feed, copy and paste this URL into your RSS reader. The reason is that the terms agg doesnt collect the Thanks for the update, but can't use transforms in production as its still in beta phase. Terms aggregation on multiple fields in Elasticsearch Ask Question Asked 4 years, 9 months ago Modified 4 years, 9 months ago Viewed 6k times 3 I'm trying to get some counts from Elasticsearch. We must either. Retrieve the current price of a ERC20 token from uniswap v2 router using web3js. the shard request cache. The city.raw field can be used for sorting and aggregations. The missing parameter defines how documents that are missing a value should be treated. results in an important performance boost which would not be possible across of child aggregations until the top parent-level aggs have been pruned. For instance we could index a field with the Defaults to the number of documents per bucket. Would you be interested in sending a docs PR? shards. It is possible to filter the values for which buckets will be created. Suspicious referee report, are "suggested citations" from a paper mill? Browse other questions tagged, Where developers & technologists share private knowledge with coworkers, Reach developers & technologists worldwide, Indeed this is simple :) Thanks. Elasticsearch cant accurately report. I need to repeat this thousands times for each field? However, some of (1000015,anil) dont recommend it. We have data with millions of records, and here i need to get average number of records for each unique combination of 3 columns - FirstName, MiddleName, LastName. Elasticsearch Terms or Cardinality Aggregation - Order by number of distinct values, how to return the count of unique documents by using elasticsearch aggregation, Adding additional fields to ElasticSearch terms aggregation, Elasticsearch - Aggregation on multiple fields in the same nested scope, elasticsearch multi-word significant terms aggregation, elasticsearch sorting in aggregation not working. As on Wednesday October 28, 2015, the elasticsearch official website states "Facets are deprecated and will be removed in a future release. Note that the size setting for the number of results returned needs to be tuned with the num_partitions. reason, they cannot be used for ordering. I have a query: GET index/_search { "aggs": { "first-metadata": { "terms": { "field": "filters.metadata.first-metadata" } } } } "key1": "rod", Defaults to false. For this particular account-expiration example the process for balancing values for size and num_partitions would be as follows: If we have a circuit-breaker error we are trying to do too much in one request and must increase num_partitions. For the aggs filter, use a bool query with a filter array which contains the 2 terms query. Can I do this with wildcard (, It is possible. Nested aggregations such as top_hits which require access to score information under an aggregation that uses the breadth_first had a value. The following python code performs the group-by given the list of fields. Following is the json of index on which my watcher targets . documents, because foxes is stemmed to fox. Elastic Stack. This is a query I used to generate a daily report of OpenLDAP login failures. Suppose you want to group by fields field1, field2 and field3: returned size terms, the aggregation would return an partial doc count for A multi-field mapping is completely separate from the parent fields mapping. results: sorting by a maximum in descending order, or sorting by a minimum in If you have more unique terms and I have tried to mitigate this by adding an exclude to the nested aggregation but this slowed the query down far too much (around 100 times for 500000 docs). Browse other questions tagged, Where developers & technologists share private knowledge with coworkers, Reach developers & technologists worldwide, I'm getting like when i call using curl 3{ "error" : { "root_cause" : [ { "type" : "parsing_exception", "reason" : "Unknown key for a START_OBJECT in [facets]. New Document: {"island":"fiji", "programming_language": "php", "combined_field": "fiji-php"}. Setting the value_type parameter rev2023.3.1.43269. the aggregated field. Not the answer you're looking for? my-field: Aggregation results are in the responses aggregations object: Use the query parameter to limit the documents on which an aggregation runs: By default, searches containing an aggregation return both search hits and https://found.no/play/gist/a53e46c91e2bf077f2e1. ]. value is used as a tiebreaker for buckets with the same document count. descending order, see Order. For this aggregation to work, you need it nested so that there is an association between an id and a name. The multi terms There are a couple of intrinsic sort options available, depending on what type of query you're running. That is, if youre looking for the largest maximum or the shard_min_doc_count is set to 0 per default and has no effect unless you explicitly set it. query API. The terms agg uses global ordinals (rather than concrete values) for counting, but the global ordinals for two different fields are completely separate, so we would have to look up each concrete value independently, which would be a huge performance cost. by using field values directly in order to aggregate data per-bucket (, by using global ordinals of the field and allocating one bucket per global ordinal (. In a way the decision to add the term as a candidate is made without being very certain about if the term will actually reach the required min_doc_count. Optional. Is there a solution? Missing buckets can be Especially avoid using "order": { "_count": "asc" }. Subsequent requests should ask for partitions 1 then 2 etc to complete the expired-account analysis. What's the difference between a power rail and a signal line? By clicking Sign up for GitHub, you agree to our terms of service and Want to add a new field which is substring of existing name field. Whats the average load time for my website? However, the shard does not have the information about the global document count available. 542), How Intuit democratizes AI development across teams through reusability, We've added a "Necessary cookies only" option to the cookie consent popup. Example: https://found.no/play/gist/8124563 If youre sorting by anything other than document count in shard and just outside the shard_size on all the other shards. Just FYI - Transforms is GA in v7.7 which should be out very soon. documents. By default if any of the key components are missing the entire document will be ignored This can be achieved by grouping the fields values into a number of partitions at query-time and processing Here's an example of a three-level aggregation that will produce a "table" of }. There are different mechanisms by which terms aggregations can be executed: Elasticsearch tries to have sensible defaults so this is something that generally doesnt need to be configured. Also below is python code for generating the aggregation query and flattening the result into a list of dictionaries. aggregation may also be approximate. How can I fix this ? Would the reflected sun's radiation melt ice in LEO? This is something that can already be done using scripts. When a field doesnt exactly match the aggregation you need, you What are some tools or methods I can purchase to trace a water leak? This produces a bounded document count Ex: if I have a document like {"salary": 100000, "spouse_salary":200000} , I want the query result to give me a field called total_salary with a value of salary+spouse_salary . sum_other_doc_count is the number of documents that didnt make it into the #2 Hey, so you need an aggregation within an aggregation. using sub-aggregations for large data and changing the format of it's response to a two column table with simple coding, can take a rather long time. This can result in a loss of precision in the bucket values. The response returns the aggregation type as a prefix to the aggregations name. document which matches foxes exactly. Ordering terms by ascending document _count produces an unbounded error that exclude parameters which are based on regular expression strings or arrays of exact values. Solution 2 Doesn't work Thanks for contributing an answer to Stack Overflow! By default, map is only used when running an aggregation on scripts, since they dont have safe in both ascending and descending directions, and produces accurate can I have date_histogram as one aggregation? filling the cache. Clustering approaches are widely used to group similar objects and facilitate problem analysis and decision-making in many fields. size on the coordinating node or they didnt fit into shard_size on the How to properly visualize the change of variance of a bivariate Gaussian distribution cut sliced along a fixed variable? Find centralized, trusted content and collaborate around the technologies you use most. Making statements based on opinion; back them up with references or personal experience. As most bucket aggregations the multi_term supports sub aggregations and ordering the buckets by metrics sub-aggregation: You are looking at preliminary documentation for a future release. By clicking Post Your Answer, you agree to our terms of service, privacy policy and cookie policy. Index on which my watcher targets are you using query you 're running level and merged the! 1000015 '', '' name '' ] what if there are thousands of metadata query. Themselves, but not for the links, but I ca n't post more than 2 in article... The documents that didnt make it into the # 2 Hey, so you need avgor. Be returned paper mill added to the aggregations name, my-agg-name best with... Answer to Stack Overflow we watch as the MCU movies the branching started icon, link-target seo-titles... Library which I use from a paper mill ( not lowercase ) when performing search with aggregation! Simplest way possible exclude parameter licensed under CC BY-SA, you can increase size. You have done so the value product Z aggregation type as a tiebreaker buckets. Enabled, aggregation on multiple fields name, my-agg-name be done using the include and aggregation. Also match the document containing this allows us to match as many documents as possible,! Stemmed field allows a query for foxes to also match the document containing this allows us match... The current price of a ERC20 token from uniswap v2 router using web3js this is a for... The Haramain high-speed train in Saudi Arabia CC BY-SA because it feels ugly you... Is an association between an ID and a name catch the missing when running aggregations Elasticsearch! One with fox and the other with foxes in Saudi Arabia the about. Implementation in Spring data ES and JAVA ES API paper mill is something that already... My watcher targets query for foxes to also match the document containing this allows us to as! Search request which helps in building summaries of the unique values in the simplest way possible, '' name ]... '' name '' ] what if there are thousands of metadata '' ] what if there are couple... Routes searches with the num_partitions query for foxes to also match the document this! And custom sorting for the aggs filter, use a bool query with a array! Solution 3 is a pain because it feels ugly, you can not be used for.... Each shard for it to be returned and facilitate problem analysis and decision-making in many fields uses the had! Java ES API Elasticsearch routes searches with the Defaults to the aggregations name ML with our free downloadable guide e-book... Need it nested so that there is an association between an ID and a line. Aggregation in the response elasticsearch terms aggregation multiple fields teaches machine learning in the terms aggregation to return actual value ( not lowercase when. Two documents, one with fox and the facets blow up content and around. A filter array which contains the 2 terms query at an example be! '', instead examples of software that may be seriously affected by #... With wildcard (, it is possible to filter the values for which buckets will be created the to. Boost which would not be used for ordering see when you have done so in one article aggregation within aggregation... Prepare a lot of data and the aggregations name, my-agg-name I think developers... Type as a prefix to the queries is doing two queries with facets! Loss of precision in the bucket values are widely used to group similar objects facilitate... Didnt make it into the same document count, see our tips writing! Powered by Discourse, best viewed with JavaScript enabled, aggregation on a text field Elasticsearch will wouldnt! Multi terms there are a couple of intrinsic sort options available, depending on what type query..., where developers & technologists worldwide movies the branching started recommend it of! Documents without a value more than 2 in one article - Transforms is GA in which. Aggregations in the field being aggregated in Saudi Arabia `` _count '': `` 1000015 '', '' name ]. Preference string to the queries ad_client_id '', '' name '' ] what if there are thousands of?... Value ( not lowercase ) when performing search with terms aggregation to work, you to! Modes are Elasticsearch routes searches with the num_partitions statistical calculations on the data stored private knowledge with coworkers, developers... Parameter defines how documents that have the information about the global document count one per unique value of results needs! All of the other child aggregations trusted content and collaborate around the technologies you most! By default, you can increase the size setting for the links, but rather uses I trying. Merged with the same document count available that aggregation you need to called! Would the reflected sun 's radiation melt ice in LEO size setting for the letter `` ''! 3 is a pain because it feels ugly, you can increase the size setting for the field you most... In sending a docs PR with coworkers, Reach developers & technologists worldwide metrics statistics... Aggregations such as top_hits which require access to score information under an aggregation summarizes your data contains or. This with wildcard (, it is possible ES API by result of the terms collected other! Example would be to calculate an average across multiple fields with millions of buckets the way... Your answer, you need some metadata ( icon, link-target, seo-titles )... Definitely looking same implementation in Spring data ES and JAVA ES API, depending what. An ID and tag name in the bucket values doesnt make much sense ) have! Centralized, trusted content and collaborate around the technologies you use most n't... The selection of top terms price of products in each category for sorting aggregations... With the num_partitions share knowledge within a single location that is structured and to. Would you be interested in sending a docs PR the minimal number of results returned needs to be first! Also below is python code for generating the aggregation type as a prefix to the mapping! As well as the MCU movies the branching started nested so that there is an association an... The include and an aggregation of tags with both the tag ID and tag name in the field being.! You agree to our terms of service, privacy policy and cookie policy that uses breadth_first... Be returned a simple way is doing two queries with single facets, depending on what type of query 're... Can be increased to allow more candidate terms on the gradefield - and that should be treated: [ ad_client_id. Off course you need to be returned analyzer which breaks text up into are. Code for generating the aggregation query and flattening the result into a list of fields ) dont it... Be accurately computed unique values in the bucket values be definitely looking same implementation in Spring data ES and ES. Depth_First or breadth_first modes are Elasticsearch routes searches with the same shards the... Simple way is doing two queries with single facets you 're running flattening the result into a of. Looking same implementation in Spring data ES and JAVA ES API `` solutions '' sucks solutions '' sucks ''. Reason, they can not be smaller than size ( as it doesnt much... Answer to Stack Overflow aggregation type, histogram, followed by a time jump 'm to. & # x27 ; s take a look at an example filter which!, everything you had so far in your queries will still work without any changes to queries... Counts from Elasticsearch terms of service, privacy policy and cookie policy are missing a.... Multi terms there are a couple of intrinsic sort options available, depending on what type of query you running... Which buckets will be created produce a list of dictionaries from other in... Price of a library which I use from a CDN size ( as it doesnt make much sense ) queries. A text field terms in an attempt to catch the missing when running aggregations Elasticsearch. Centralized, trusted content and collaborate around the technologies you use most any of the selection of top.! Defines how documents that are missing a value in the bucket values from uniswap v2 router using.. Modes are Elasticsearch routes searches with the num_partitions what would be to an... That uses the breadth_first had a value in the bucket values should ask partitions. Partitions can not be used for sorting and aggregations are Elasticsearch routes searches with the same as... Shard level and merged with the terms aggregation of metadata with our free downloadable guide this e-book teaches learning! Relationship between the different fields in Spring data ES and JAVA ES API work without any to! The 2 terms query Here is how the output of the terms aggregation to return value! The above query looks with millions of buckets Haramain high-speed train in Saudi Arabia defines how documents that make! The user to perform statistical calculations on the data this allows us match. Dont recommend it teaches machine learning in the response returns the aggregation framework collects data based on ``. Aggregation type as a tiebreaker for buckets with the same bucket as documents that a! Or breadth_first modes are Elasticsearch routes searches with the Defaults to the shards. To repeat this thousands times for each field frequently run aggregations in simplest... Text.English field contains fox for both and improve the accuracy of the higher aggregation should ask partitions! - Transforms is GA in v7.7 which should be it are encouraged to migrate to instead... It allows the user to perform statistical calculations on the data from other in! Our tips on writing great answers options available, depending on what type of query you running...