• Home   /  
  • Archive by category "1"

Mongodb For Dba Homework 3.4

It's not an easy task to plan the dimensions of shards, and there is no fool or errorproof way of doing it.

What I tend to do is to create a certain fraction of the expected data per shard as dummy data. If I have a shard handy, I fill it completely.

In the next step, I generate the same fraction of the expected load on a matching number of front-end servers.

Now, I analyse the memory consumption of the indices and overall. Knowing the index size, you roughly know the minimum RAM you need. In order to have enough warning time and a decent amount of RAM for connections, the working set and other processes, I round up to the next sensible value for RAM sizes. This way you make sure your RAM does not get filled up easily, you have a decent warning time when you have too scale out because of insufficient RAM and your application won't be throttled by your database.

Let me illustrate that. Given my average object size is slightly under 1kB and I plan to have a 1TB partition, but I only have a 256GB partition for tests, I'd create somewhere in the order of 250 million dummy documents. Now, let's say we plan to have 8 front-end servers for the application, I'd set up two of them.

Next, I'd use a load testing tool (there are many, and they are out of scope of this answer) to generate load on those two front-end servers.

After you have done that, you should have a quite accurate indication of overall memory consumption, index size and working set size.

Let's say we find out that our indices are 20GB in size and 300 connections are made. So, let's just right say 21GB. So our shard should have four times this RAM, rounded up to the next sensible value (keep in mind cost efficiency!) So, let's assume that would be 96GB.

Now, you should find out wether it makes more sense to have two shards instead of one, each with 48GB of RAM and 500GB of disk space for MongoDB, respectively. This may be the case, because bigger SSDs can be disproportionately expensive.

I think you get the picture: There is no easy formula, you simply have to do your homework ;) My shortish might or might not work for you. If it doesn't, you have to work one out yourself.

Oh, and whatever you do when planning a shard: plan with SSDs. It's worth it.

Check your connection pool size information with connPoolStats

From there you can see what is f.ex. max possible connection count. That depend how much your server have memory.

And with connection string you can set max and min poolsize. 100 is default value.

About current connections, shows information. For production systems it is typical to adjust the ulimit settings on Linux to allow more concurrent connections. For more best practices, I would recommend reviewing the Production Notes in the MongoDB manual.

But you still need to change your values before you can go over that 100 connections.

net.maxIncomingConnections
Type: integer

Default: 65536

The maximum number of simultaneous connections that mongos or mongod will accept. This setting has no effect if it is higher than your operating system’s configured maximum connection tracking threshold.

Total connections available tells how many connections that node can have. tell how much that connection pool can have. You need to change your pool size.

One thought on “Mongodb For Dba Homework 3.4

Leave a comment

L'indirizzo email non verrà pubblicato. I campi obbligatori sono contrassegnati *