I will need to create a simple analysis back-end to capture user behavior, such as webpage such as Google Analytics or MixPlan data But will be captured via a javascript snippet.
The system has to capture pass-to-real-time browser data (scrolling position, mouse position etc. of the page), it will record users' pages every 5 seconds. There are only three attributes on each measure but they should be taken often
No need to send the data in every 5 seconds, it can be done less frequently but it is necessary that all of us on the user's page Find the data. That is, I can not just sit it once per minute and lost the last 59 seconds of data for anyone who left after 119 seconds.
If possible, I would like to create a system which is a scale for the near future which means that it is working for 10,000 sites, with 100 concurrent visitors in each, i.e. 100,000 concurrent Users who send an event every 5 seconds
I'm not worried about data inquiries, which can be done using a different system. I am most interested in how to handle data capture
requirements
Based on the above budget, the system needs to handle 20,000 events per second coming from the pool of 100,000 users is.
I want to host this service while hosting my service on Hirok, I have done a lot with the railways, I am totally new to the high throttle system (except knowing that you can use them by rail Do not process).
Question - Is there a business system that would be good to do this (like Pushar, but with Data Capture as well as Distribution)?
- Do I request it to http requests or websites?
- Is this the right choice for this node or simply trendy?
- If I have chosen a socket based solution, then how many sockets can do a dino on Heroku, handle for each webserver
- between storage Mongo / Reddis etc What are the proper thoughts to choose from?
- Is it a problem that really requires two solutions - to get you the first to get the right scale early and unexpected Y and the other will be less incremental on that scale L It should be taken for carrying on the path, but what is necessary before more development efforts?
You have to build your system after designing my high level of comment, and Then worry about scaling such as customers arrive i node. I am thrilled with the JS and NMM ecosystem, but I also think that you can make a fully acceptable platform with rail. If 100 k with nodes take 3 dyes to support concurrent users, and double with the rail, then you can still be better with Ruby, if your comfort with Ruby gives you 3 months faster Found in the market. Anyway, you agree to go with the node, here's my answer:
- Here are some of Pushar who can work for you and discuss. See also.
- Use. This is a fair standard, because it uses the best available transport and comes back from WebSockets in HTTP methods.
- The node is a great option and it is also fashionable (see module). I suspect that you can work properly in node, rail or many other frameworks.
- Depending on how efficient the RAM relies on, a Horoko witch should be able to support thousands of concurrent connections. A 16 GB RAM server was able to support a concurrent connection. Let's say that you are RAM-limited, that Heroica Dion with 512 MB RAM should be able to support ~ 30K connection
- You probably want to choose two different systems, one for storing and processing your data, and one for caching. Here's a great deal about choosing your core data platform from the creator of Instagram. For core data, I recommend postgraze (herooco) using sequelic ORM. But, Mongo with SOLR will probably work well for search. Note that Postgres 9.2 can be used as a NoSUl Datastore, if you want to go the way I recommend the excessive redundancy for the caching system. No, I will try to avoid throwing engineering instead, do some work that works, and hopes that every time you reach the maximum traffic volume, the system's Some parts will be broken and it will need to be changed but, if you follow 12 factor principles, you should be in good condition to scale horizontally while investing in the replacement.
Good luck.
You have to build your system after designing my high level of comment, and Then worry about scaling such as customers arrive i node. I am thrilled with the JS and NMM ecosystem, but I also think that you can make a fully acceptable platform with rail. If 100 k with nodes take 3 dyes to support concurrent users, and double with the rail, then you can still be better with Ruby, if your comfort with Ruby gives you 3 months faster Found in the market. Anyway, you agree to go with the node, here's my answer:
- Here are some of Pushar who can work for you and discuss. See also.
- Use. This is a fair standard, because it uses the best available transport and comes back from WebSockets in HTTP methods.
- The node is a great option and it is also fashionable (see module). I suspect that you can work properly in node, rail or many other frameworks.
- Depending on how efficient the RAM relies on, a Horoko witch should be able to support thousands of concurrent connections. A 16 GB RAM server was able to support a concurrent connection. Let's say that you are RAM-limited, that Heroica Dion with 512 MB RAM should be able to support ~ 30K connection
- You probably want to choose two different systems, one for storing and processing your data, and one for caching. Here's a great deal about choosing your core data platform from the creator of Instagram. For core data, I recommend postgraze (herooco) using sequelic ORM. But, Mongo with SOLR will probably work well for search. Note that Postgres 9.2 can be used as a NoSUl Datastore, if you want to go the way I recommend the excessive redundancy for the caching system. No, I will try to avoid throwing engineering instead, do some work that works, and hopes that every time you reach the maximum traffic volume, the system's Some parts will be broken and it will need to be changed but, if you follow 12 factor principles, you should be in good condition to scale horizontally while investing in the replacement.
Good luck.
No comments:
Post a Comment