Expire Data from Collections by Setting TTL

Expire Data from Collections by Setting TTL

It is new in version 2.2. and the MongoDB’s “time to live” or “TTL” collection feature make it possible to store data in MongoDB and have the mongod automatically remove data after a specified number of seconds or at a specific clock time. Data expiration is useful for some classes of information, including machine generated event data, logs, and session information that only need to persist for a limited period of time.

A special index type supports the implementation of TTL collections. TTL relies on a background thread in mongod that reads the date-typed values in the index and removes expired documents from the collection.

Considerations

• The _id field does not support TTL indexes.
• You cannot create a TTL index on a field that already has an index.
• A document will not expire if the indexed field does not exist.
• A document will not expire if the indexed field is not a date BSON type or an array of date BSON types.
• The TTL index may not be compound (may not have multiple fields).
• If the TTL field holds an array, and there are multiple date-typed data in the index, the document will expire when the lowest (i.e. earliest) date matches the expiration threshold.
• You cannot create a TTL index on a capped collection, because MongoDB cannot remove documents from a capped collection.
• You cannot use ensureIndex() to change the value of expireAfterSeconds. Instead use the collMod database command in conjunction with the index collection flag.
• When you build a TTL index in the background, the TTL thread can begin deleting documents while the index is building. If you build a TTL index in the foreground, MongoDB begins removing expired documents as soon as the index finishes building.

When the TTL thread is active, you will see delete operations in the output of db.currentOp() or in the data collected by the database profiler. When using TTL indexes on replica sets, the TTL background thread only deletes documents on primary members. However, the TTL background thread does run on secondaries. Secondary members replicate deletion operations from the primary.

The TTL index does not guarantee that expired data will be deleted immediately. There may be a delay between the time a document expires and the time that MongoDB removes the document from the database. The background task that removes expired documents runs every 60 seconds. As a result, documents may remain in a collection after they expire but before the background task runs or completes.

The duration of the removal operation depends on the workload of your mongod instance. Therefore, expired data may exist for some time beyond the 60 second period between runs of the background task. All collections with an index using the expireAfterSeconds option have usePowerOf2Sizes enabled. Users cannot modify this setting. As a result of enabling usePowerOf2Sizes, MongoDB must allocate more disk space relative to data size. This approach helps mitigate the possibility of storage fragmentation caused by frequent delete operations and leads to more predictable storage use patterns.

Procedures – To enable TTL for a collection, use the ensureIndex() method to create a TTL index, as shown in the examples below. With the exception of the background thread, a TTL index supports queries in the same way normal indexes do. You can use TTL indexes to expire documents in one of two ways, either:

• remove documents a certain number of seconds after creation. The index will support queries for the creation time of the documents. Alternately,
• specify an explicit expiration time. The index will support queries for the expiration-time of the document.

Expire Documents after a Certain Number of Seconds – To expire data after a certain number of seconds, create a TTL index on a field that holds values of BSON date type or an array of BSON date-typed objects and specify a positive non-zero value in the expireAfterSeconds field. A document will expire when the number of seconds in the expireAfterSeconds field has passed since the time specified in its indexed field. For example, the following operation creates an index on the log_events collection’s createdAt field and specifies the expireAfterSeconds value of 3600 to set the expiration time to be one hour after the time specified by createdAt.

db.log_events.ensureIndex( { “createdAt”: 1 }, { expireAfterSeconds: 3600 } )

When adding documents to the log_events collection, set the createdAt field to the current time:

db.log_events.insert( {

“createdAt”: new Date(),

“logEvent”: 2,

“logMessage”: “Success!”

} )

MongoDB will automatically delete documents from the log_events collection when the document’s createdAt value is older than the number of seconds specified in expireAfterSeconds.

Expire Documents at a Certain Clock Time – To expire documents at a certain clock time, begin by creating a TTL index on a field that holds values of BSON date type or an array of BSON date-typed objects and specify an expireAfterSeconds value of 0. For each document in the collection, set the indexed date field to a value corresponding to the time the document should expire. If the indexed date field contains a date in the past, MongoDB considers the document expired. For example, the following operation creates an index on the app.events collection’s expireAt field and specifies the expireAfterSeconds value of 0:

db.app.events.ensureIndex( { “expireAt”: 1 }, { expireAfterSeconds: 0 } )

For each document, set the value of expireAt to correspond to the time the document should expire. For instance, the following insert() operation adds a document that should expire at July 22, 2013 14:00:00.

db.app.events.insert( {

“expireAt”: new Date(‘July 22, 2013 14:00:00’),

“logEvent”: 2,

“logMessage”: “Success!”

} )

MongoDB will automatically delete documents from the app.events collection when the documents’ expireAt value is older than the number of seconds specified in expireAfterSeconds, i.e. 0 seconds older in this case. As such, the data expires at the specified expireAt value.

Apply for MongoDB Certification Now!!

https://www.vskills.in/certification/databases/mongodb-server-administrator

Back to Tutorial