ArangoDB: Working with documents

Document store the data in the form of JSON objects. Every document has a key and handle.

 

Document key

Document key is a string used to uniquely identify a document within the collection. Document key is stored in _key attribute of the document. Once _key attribute is assigned with a value, it is immutable, its value is not going to be changed any time.

 

Is the document key values are indexed?

Yes

 

Can I provide custom value to _key attribute?

Yes, you can. But you need to make sure that the _key value is unique in the collection and it is immutable.

 

Document Handle

Document handle is used to uniquely identify a document within the database.  It is a string consist of collection name followed by / followed by the value of _key attribute.

 

Example

127.0.0.1:8529@abc_org> userCollection.save({"id" : 1, "firstName": "Ram"})
{ 
  "_id" : "user/12977", 
  "_key" : "12977", 
  "_rev" : "_cRdSijO---" 
}

In the above example, 12977 is the Document key and "user/12977" is the document handle.

 

Document Revision

Every document has a revision specified using _rev attribute. _rev attribute is read-only to the user. It is calculated by ArangoDB server. Whenever a document is updated, this revision value is going to be changed.

 

For example, lets create a document.

127.0.0.1:8529@abc_org> userCollection.save({"lastName" : "Gurram", "firstName": "Ram"})
{ 
  "_id" : "user/14434", 
  "_key" : "14434", 
  "_rev" : "_cRd20V6---" 
}

Let’s update the document and confirm that the revision is updated.

127.0.0.1:8529@abc_org> userCollection.update("user/14434", {"lastName": "Battu"}, false, false)
{ 
  "_id" : "user/14434", 
  "_key" : "14434", 
  "_rev" : "_cRd4ppe---", 
  "_oldRev" : "_cRd20V6---" 
}

You can observe that new revision is updated. After some time old revision content is deleted.

 

Can I perform operations on multiple documents using single command?

Yes, from ArangoDB 3.0 onwards, you can perform actions on multiple documents in a single command.

 

      ArangoDB: all(): Get all the documents in a collection

      ArangoDB: Convert the cursor to array

      ArangoDB: Traverse a cursor using next and hasNext methods

      ArangoDB: limit(): limit number of documents to be returned

      ArangoDB: skip(): skip number of documents

      ArangoDB: Query by example

      ArangoDB: firstExample: Get one document that matches to given example

      ArangoDB: range, closedRange: Get all the documents where attribute is in given range

      ArangoDB: any: Get any one document

      ArangoDB: count: Count number of live documents

      ArangoDB: toArray: Convert the collection into an array

      ArangoDB: Find the document by id or key

      ArangoDB: Get multiple documents in one call

      ArangoDB: Executing queries with arangosh

      ArangoDB: exists: Check whether document exists or not

      ArangoDB: Check for the existence of multiple documents

      ArangoDB: Get documents by keys

      ArangoDB: insert/save: Insert new documents to the collection

      ArangoDB: Replace the existing document

      ArangoDB: update: Update the content of a document

      ArangoDB: Remove a document

      ArangoDB: Remove documents by keys

      ArangoDB: Remove by example

      ArangoDB: Replace by Example

      ArangoDB: Update the documents by example

      ArangoDB: Get the type of collection

      ArangoDB: Get document id from key

      ArangoDB: Working with edge documents

      ArangoDB: Enforce schema at collection level

 

 

 

Previous                                                    Next                                                    Home

ArangoDB: truncate: remove all the documents but keep indexes

‘truncate’ method is used to remove all the documents from a collection (it do not remove the indexes associated with the collection).

 

There are two ways to remove a collection.

a.   Using truncate method of collection

b.   Using truncate method of db

 

Using truncate method of collection

Syntax

collection.truncate()

 

Step 1: Let’s create a collection demodb by executing below statement.

db._create("demodb")

127.0.0.1:8529@demo> db._create("demodb")
[ArangoCollection 10954, "demodb" (type document, status loaded)]

127.0.0.1:8529@demo> db._collections()
[ 
  [ArangoCollection 10043, "_analyzers" (type document, status loaded)], 
  [ArangoCollection 10058, "_appbundles" (type document, status loaded)], 
  [ArangoCollection 10055, "_apps" (type document, status loaded)], 
  [ArangoCollection 10046, "_aqlfunctions" (type document, status loaded)], 
  [ArangoCollection 10067, "_fishbowl" (type document, status loaded)], 
  [ArangoCollection 10061, "_frontend" (type document, status loaded)], 
  [ArangoCollection 10040, "_graphs" (type document, status loaded)], 
  [ArangoCollection 10052, "_jobs" (type document, status loaded)], 
  [ArangoCollection 10064, "_modules" (type document, status loaded)], 
  [ArangoCollection 10049, "_queues" (type document, status loaded)], 
  [ArangoCollection 10954, "demodb" (type document, status loaded)] 
]

 

Step 2: Let’s add some content to demodb collection by executing below statements.

col = db.demodb

col.save({"id" : 1, "name" : "Krishna"})

 

Count number of elements in the collection using ‘count’ method.

col.count()

 

127.0.0.1:8529@demo> col = db.demodb
[ArangoCollection 10954, "demodb" (type document, status loaded)]

127.0.0.1:8529@demo> col.save({"id" : 1, "name" : "Krishna"})
{ 
  "_id" : "demodb/11019", 
  "_key" : "11019", 
  "_rev" : "_cRclYT2---" 
}

127.0.0.1:8529@demo> col.count()
1

  Truncate the collection elements and confirm the same using count method.

 

127.0.0.1:8529@demo> col.truncate()

127.0.0.1:8529@demo> col.count()
0

 

Using truncate method of db

Signature

db._truncate(collection-identifier)

db._truncate(collection-name)

 

Truncate a collection using id

Step 1: Create a collection with name test.

127.0.0.1:8529@demo> db._create("test")
[ArangoCollection 11517, "test" (type document, status loaded)]

127.0.0.1:8529@demo> db._collections()
[ 
  [ArangoCollection 10043, "_analyzers" (type document, status loaded)], 
  [ArangoCollection 10058, "_appbundles" (type document, status loaded)], 
  [ArangoCollection 10055, "_apps" (type document, status loaded)], 
  [ArangoCollection 10046, "_aqlfunctions" (type document, status loaded)], 
  [ArangoCollection 10067, "_fishbowl" (type document, status loaded)], 
  [ArangoCollection 10061, "_frontend" (type document, status loaded)], 
  [ArangoCollection 10040, "_graphs" (type document, status loaded)], 
  [ArangoCollection 10052, "_jobs" (type document, status loaded)], 
  [ArangoCollection 10064, "_modules" (type document, status loaded)], 
  [ArangoCollection 10049, "_queues" (type document, status loaded)], 
  [ArangoCollection 11517, "test" (type document, status loaded)] 
]

 

Step 2: Add some data to the test collection.

127.0.0.1:8529@demo> colId = db.test
[ArangoCollection 11517, "test" (type document, status loaded)]

127.0.0.1:8529@demo> colId.save({"id" : 1, "firstName": "Krishna"})
{ 
  "_id" : "test/11588", 
  "_key" : "11588", 
  "_rev" : "_cRczMnm---" 
}


127.0.0.1:8529@demo> colId.count()
1

 

Step 3: truncate the collection data using collection identifier.

127.0.0.1:8529@demo> db._truncate(colId)

127.0.0.1:8529@demo> colId.count()
0

 

Truncate a collection using collection name

Step 1: Add some data to the test collection.

127.0.0.1:8529@demo> colId = db.test
[ArangoCollection 11517, "test" (type document, status loaded)]

127.0.0.1:8529@demo> colId.save({"id": 123, "firstName" : "Sailu"})
{ 
  "_id" : "test/11683", 
  "_key" : "11683", 
  "_rev" : "_cRc1Yd----" 
}

127.0.0.1:8529@demo> colId.count()
1

 

Step 2: Truncate the contents of collection.

127.0.0.1:8529@demo> db._truncate("test")

127.0.0.1:8529@demo> colId.count()
0

 

 

 

 

 

 

 

 

 

Previous                                                    Next                                                    Home

Lucene: Update a document

Lucene provides 'updateDocument', 'updateDocuments' method to update the documents in index.

 

Signature

public long updateDocument(Term term, Iterable<? extends IndexableField> doc) throws IOException

This method updates a document by first deleting the document(s) containing term and then adding the new document. 

 

public long updateDocuments(Term delTerm, Iterable<? extends Iterable<? extends IndexableField>> docs) throws IOException

Atomically deletes documents matching the provided delTerm and adds a block of documents with sequentially assigned document IDs, such that an external reader will see all or none of the documents.

 

Example

Term term = new Term("id", "2");
indexWriter.updateDocument(term, newDoc);

 

DocumentUtil.java

package com.sample.app.util;

import java.util.Arrays;
import java.util.List;

import org.apache.lucene.document.Document;
import org.apache.lucene.document.Field;
import org.apache.lucene.document.TextField;

public class DocumentUtil {

	public static Document getDocument(String id, String title, String description, String blog) {
		Document doc = new Document();
		doc.add(new TextField("id", id, Field.Store.YES));
		doc.add(new TextField("title", title, Field.Store.YES));
		doc.add(new TextField("description", description, Field.Store.NO));
		doc.add(new TextField("blog", blog, Field.Store.YES));
		
		return doc;

	}

	public static List<Document> getDocuments() {
		Document doc1 = getDocument("1", "JavaWorld",
				"The original independent resource for Java developers, architects, and managers.", " javaworld.com");
		Document doc2 = getDocument("2", "Oracle Blogs | The Java Source",
				" Java powers more than 4.5 billion devices including 800 million computers and 1.5 billion cell phones. If you love Java, this is the blog you must follow.",
				"blogs.oracle.com/java");
		Document doc3 = getDocument("3", "A Java geek",
				"Nicolas Fränkel's blog. IT architect focusing on Java, Java EE, and their surrounding ecosystems. He is a trainer, book writer, speaker & blogger.",
				"blog.frankel.ch");
		Document doc4 = getDocument("4", "Self Learning Java", "Learn Java fundamentals and other java libraries",
				"self-learning-java-tutorial.blogspot.com");

		return Arrays.asList(doc1, doc2, doc3, doc4);

	}
}

 

App.java

package com.sample.app;

import java.io.File;
import java.io.IOException;
import java.util.List;

import org.apache.lucene.analysis.Analyzer;
import org.apache.lucene.analysis.standard.StandardAnalyzer;
import org.apache.lucene.document.Document;
import org.apache.lucene.index.DirectoryReader;
import org.apache.lucene.index.IndexReader;
import org.apache.lucene.index.IndexWriter;
import org.apache.lucene.index.IndexWriterConfig;
import org.apache.lucene.index.MultiBits;
import org.apache.lucene.index.Term;
import org.apache.lucene.search.IndexSearcher;
import org.apache.lucene.search.Query;
import org.apache.lucene.search.ScoreDoc;
import org.apache.lucene.search.TermQuery;
import org.apache.lucene.search.TopDocs;
import org.apache.lucene.store.Directory;
import org.apache.lucene.store.MMapDirectory;
import org.apache.lucene.store.NoLockFactory;
import org.apache.lucene.util.Bits;

import com.sample.app.util.DocumentUtil;

public class App {

	private static void printAllDocuments(Directory directory) throws IOException {

		try (IndexReader indexReader = DirectoryReader.open(directory)) {
			System.out.println("All Documents in Lucene Index");
			Bits liveDocs = MultiBits.getLiveDocs(indexReader);
			for (int i = 0; i < indexReader.maxDoc(); i++) {
				if (liveDocs != null && !liveDocs.get(i))
					continue;

				Document doc = indexReader.document(i);
				System.out.println(doc.get("id") + ", " + doc.get("title"));
			}

			System.out.println();
		}

	}

	public static void main(String args[]) throws IOException {

		Analyzer analyzer = new StandardAnalyzer();
		IndexWriterConfig indexWriterConfig1 = new IndexWriterConfig(analyzer);

		Directory directory = new MMapDirectory(new File("/Users/Shared/lucene").toPath(), NoLockFactory.INSTANCE);

		try (IndexWriter indexWriter = new IndexWriter(directory, indexWriterConfig1)) {

			List<Document> documents = DocumentUtil.getDocuments();

			System.out.println("Adding " + documents.size() + " documents to Lucene");
			indexWriter.addDocuments(documents);
			indexWriter.commit();

			printAllDocuments(directory);

			Term term = new Term("id", "2");

			try (IndexReader indexReader = DirectoryReader.open(directory)) {
				IndexSearcher indexSearcher = new IndexSearcher(indexReader);
				Query query = new TermQuery(term);

				TopDocs docs = indexSearcher.search(query, 1);
				ScoreDoc[] hits = docs.scoreDocs;

				Document document = indexSearcher.doc(hits[0].doc);

				Document newDoc = DocumentUtil.getDocument(document.get("id"), "Java Revisited",
						"Java Design Patterns and Java interview Questions", "https://javarevisited.blogspot.com/");

				indexWriter.updateDocument(term, newDoc);
			}

		}
		printAllDocuments(directory);

	}
}

 

Output

Adding 4 documents to Lucene
All Documents in Lucene Index
1, JavaWorld
2, Oracle Blogs | The Java Source
3, A Java geek
4, Self Learning Java

All Documents in Lucene Index
1, JavaWorld
3, A Java geek
4, Self Learning Java
2, Java Revisited

 

 

  

Previous                                                    Next                                                    Home

Lucene: Check any document is deleted as part of delete query or not

'indexWriter.hasDeletions()' method return true if this index has deletions (including buffered deletions).

 

DocumentUtil.java

package com.sample.app.util;

import java.util.Arrays;
import java.util.List;

import org.apache.lucene.document.Document;
import org.apache.lucene.document.Field;
import org.apache.lucene.document.TextField;

public class DocumentUtil {

	private static Document getDocument(String id, String title, String description, String blog) {
		Document doc = new Document();
		doc.add(new TextField("id", id, Field.Store.YES));
		doc.add(new TextField("title", title, Field.Store.YES));
		doc.add(new TextField("description", description, Field.Store.NO));
		doc.add(new TextField("blog", blog, Field.Store.YES));
		
		return doc;

	}

	public static List<Document> getDocuments() {
		Document doc1 = getDocument("1", "JavaWorld",
				"The original independent resource for Java developers, architects, and managers.", " javaworld.com");
		Document doc2 = getDocument("2", "Oracle Blogs | The Java Source",
				" Java powers more than 4.5 billion devices including 800 million computers and 1.5 billion cell phones. If you love Java, this is the blog you must follow.",
				"blogs.oracle.com/java");
		Document doc3 = getDocument("3", "A Java geek",
				"Nicolas Fränkel's blog. IT architect focusing on Java, Java EE, and their surrounding ecosystems. He is a trainer, book writer, speaker & blogger.",
				"blog.frankel.ch");
		Document doc4 = getDocument("4", "Self Learning Java", "Learn Java fundamentals and other java libraries",
				"self-learning-java-tutorial.blogspot.com");

		return Arrays.asList(doc1, doc2, doc3, doc4);

	}
}

 

App.java

package com.sample.app;

import java.io.File;
import java.io.IOException;
import java.util.List;

import org.apache.lucene.analysis.Analyzer;
import org.apache.lucene.analysis.standard.StandardAnalyzer;
import org.apache.lucene.document.Document;
import org.apache.lucene.index.DirectoryReader;
import org.apache.lucene.index.IndexReader;
import org.apache.lucene.index.IndexWriter;
import org.apache.lucene.index.IndexWriterConfig;
import org.apache.lucene.index.MultiBits;
import org.apache.lucene.index.Term;
import org.apache.lucene.store.Directory;
import org.apache.lucene.store.MMapDirectory;
import org.apache.lucene.store.NoLockFactory;
import org.apache.lucene.util.Bits;

import com.sample.app.util.DocumentUtil;

public class App {

	private static void printAllDocuments(Directory directory) throws IOException {

		try (IndexReader indexReader = DirectoryReader.open(directory)) {
			System.out.println("All Documents in Lucene Index");
			Bits liveDocs = MultiBits.getLiveDocs(indexReader);
			for (int i = 0; i < indexReader.maxDoc(); i++) {
				if (liveDocs != null && !liveDocs.get(i))
					continue;

				Document doc = indexReader.document(i);
				System.out.println(doc.get("id") + ", " + doc.get("title"));
			}

			System.out.println();
		}

	}

	public static void main(String args[]) throws IOException {

		Analyzer analyzer = new StandardAnalyzer();
		IndexWriterConfig indexWriterConfig1 = new IndexWriterConfig(analyzer);

		Directory directory = new MMapDirectory(new File("/Users/Shared/lucene").toPath(), NoLockFactory.INSTANCE);

		try (IndexWriter indexWriter = new IndexWriter(directory, indexWriterConfig1)) {

			List<Document> documents = DocumentUtil.getDocuments();

			System.out.println("Adding " + documents.size() + " documents to Lucene");
			indexWriter.addDocuments(documents);
			indexWriter.commit();

			printAllDocuments(directory);

			System.out.println("\nAbout to delete all documents");

			indexWriter.deleteDocuments(new Term("id", "2"));
			indexWriter.commit();

			System.out.println("Has Deletions : " + indexWriter.hasDeletions());

			printAllDocuments(directory);
		}

	}
}

 

Output

Adding 4 documents to Lucene
All Documents in Lucene Index
1, JavaWorld
2, Oracle Blogs | The Java Source
3, A Java geek
4, Self Learning Java


About to delete all documents
Has Deletions : true
All Documents in Lucene Index
1, JavaWorld
3, A Java geek
4, Self Learning Java

 

 

 

 

 

Previous                                                    Next                                                    Home