I have a asp.net 4.0 web application that uses linqtosql to insert/update/delete data from a sql server 2005 database.
The application is elastic and therefore fits on any device... phone, ipad or desktop.
On phones and ipads users will normally have a data plan.
What I want to know is when a user is connected to my application, but not inserting/updating/deleting data, is it affecting data usage?
Also is there data usage if a user is just looking at pages or does data usage only occur when data is being inserted/updated/deleted?
Thanks
I can't tell if you're trolling or not...
But if you aren't, then you don't understand how web applications work. If it appears in the browser window then it means the user agent (the browser in the device) downloaded it from the web server, that download eats into the device's data allowance. The fact that data was possibly moved around on the server side is irrelevant.
When a web user agent (i.e. a browser) submit data back to the server (usually via a POST request) then that too will eat into the data allowance, but HTML forms typically consume less than a kilobyte of data when being sent from the client (unless you're uploading binary files or large textareas). Data allowances are usually bi-directional, so you might have 1GB to use for both upstream and downstream - the few kilobytes you use uploading a form's data are massively outweighed by the fact that most webpages thesedays weigh a few hundred kilobytes, if not more.
Of course, the right answer in this case is "profile!" - run your application yourself and see how much data you go through when performing common tasks, record your findings, and act accordingly.
Related
I am working on an ASP.NET webforms application with Entity Framework. Also for some reports it uses a dll and in that we have explicit query to get the records from SQL Server (such as ADO).
The problem is that when I change a column such as ParentID in SQL Server, I must to reset the website in IIS to see it and this solves the problem. This dependency is not logical and I want to know why this happens? Is there any relation to caching because of calling method in the dll?
How can I solve this problem?
When you run a query against SQL server (or any database, really), the result that you see is not the data "in the database", so to speak. The query returns a copy of that data that belongs only to you. The copy of the data gets sent over the network, to the client - in your case, an ASP.NET web application - and the application does whatever it needs to do, such as show it to a user.
Once the query which retrieved the data is complete, there is no longer any link between the data in the client, and the data in the database. There is no continuous, "live" connection between the two, even if your actual database connection is still open. The database connection is merely a way to send queries to the server, and for it to send copies of the data back.
It's like taking a copy of a file from a different machine. If you copy a file from my machine, and then I update my copy, your copy doesn't instantly get updated.
If you want data in some user interface to stay perfectly up to date with the data that actually exists in the database, you have a difficult problem to solve. There is no "easy" way to do this. Or perhaps more accurately, there is no simple or efficient way to do this.
This might seem odd to you. You're thinking "well, why not? Why doesn't it just show me the values as they actually exist?". The reason is that these systems need to be able to support many users - often thousands at once - who are all both reading the database and writing to it. Imagine someone was in the middle of updating data in the database, but then they rollback their transaction. Should you see the data as it was being modified, but not committed? What if two users are trying to update "the same" data at once? All sorts of concurrency questions come into play, which basically boils down to questions about locking.
What you are encountering here is a basic principle of multi-threaded environments, which translates to systems with multiple clients: Data can't be accessed directly by multiple people at the same time. Instead, you give each person their own immutable copy.
In a web application things are even more disconnected. When the browser requests the web page, the server side of the web application gets a copy of the data from the database, and then transmits that to the browser. Once the page is loaded there is no longer any link between the web server and the database server, or any link between the web server and the web browser at the client, and certainly no link between the web browser and the database.
Ultimately, this is one of the "hard problems" in computer science. You want to know how to tell the client to invalidate their "cache", and refresh their local data. There are a few mechanisms provided by .NET to do this with SQL Server, but they are quite technical. One of them is query notifications
Can someone give me a good reason why ViewState isn't stored on the server by default?
Why not send a small session token in place of ViewState, which can then be mapped to whatever ViewState info is needed on the server, to prevent the whole ViewState being posted back and forwards multiple times.
Am I missing something?
Scalability - imagine how much server resources would be needed if a complex WebForms page was viewed by 1M users. Server would need to hold ViewState for at least the duration of the session timeout. Automatic server side cleanup of viewstate would also be problematic - user may be viewing several pages at once so ViewState for all pages would need to be retained.
Edit
There are several techniques discussed in these posts on how to move viewstate to the server. However, before you do that, it would be a good idea to remove unnecessary viewstate from controls / pages which don't need it (e.g. View only / no postback rendering).
I'm guessing now, but when viewstate was designed 10 years or so ago, 1GB RAM on a 32 bit server was about as good as it got, and MS presumably had to think of hosting providers wanting load 100's of apps per server. So bandwidth was probably viewed as cheaper than server Ram and disk storage.
There are a number of issues with storing the ViewState in memory.
If the application recycles, the VS for all anyone using the application is lost.
It increases the memory consumption of the application. This isn't an issue if there are only a few apps hosted on a server; but there are cases when there could be many websites hosted on one box.
Scalability; the more active the application, the more VS needs to be stored. And you can't assume 1-1 (1 user - 1 VS). A user can have multiple tabs open, can go back, leave tabs inactive, etc... which leads to:
How long do you store VS? Keeping the data encoded on the page ensures that it'll still be there if the user leaves the site open for a while.
What happens if you're hosted on a web farm. We can't guarantee that the user will hit the same machine on each request.
That being said, there are a few solutions:
Memcached-Viewstate - stores the VS in distributed memory using Memcache. This isn't ideal - if a server goes down the VS for anyone who had the VS stored to that server is lost, but will allow for application pools to reset without issue.
SQL-Viewstate - stores the VS in a SQL database. This adds a least 1 DB read and 1 DB write per request. Again, not ideal, but if the VS is getting unmanagable getting and setting the VS from the database is faster than sending and recieving it over HTTP.
Filesystem-Viewstate - stores the VS in the filesystem. It's less expensive than the SQL connection but would require a file server to work in a distributed environment.
It improves scalability because the server doesn't need to maintain all of that in memory. It is possible to store the viewstate in session but it's generally not recommended.
The root cause is using client side view state is that server doesn't know the current state of the page.
If a user is anxious, does multiple (partial) postback on the page, without waiting the response, browser will send out multiple partial postback requests, that each request create a new view state on server side, which will eventually flush out the initial view state in the browser. Finally the user does his last postback, at that time, the inital copy is gone, thus exception is thrown.
Also server side view state impacts server performance and user experience. If a user doesn't interact with the page for a day or a long time, the view state on server will expire. When the user posts back the page later, an exception is thrown.
For instance I watch youtube video of length 40 minutes. Yesterday I watched the first half, didn't close the tab but hiberated my computer. Today I continue watchig the last half, and post back something, the page will get errored out if the view state is in server and expired.
We need to add a feature to our website that allows the user to test his connection speed (upload/download). After some research I found that the way this being done is downloading/uploading a file and divide the file size by the time required for that task, like here http://www.codeproject.com/KB/IP/speedtest.aspx
However, this is not possible in my case, this should be a web application so I can't download/upload files to/from user's machine for obvious security reasons. From what I have read this download/upload tasks should be on the server, but how?
This website has exactly what I have in mind but in php http://www.brandonchecketts.com/open-source-speedtest. Unfortunately, I don't have any background about php and this task is really urgent.
I really don't need to use stuff like speedtest
thanks
The formula for getting the downloadspeed in bit/s:
DataSize in byte * 8 / (time in seconds to download - latency to the server).
Download:
Host a blob of data (html document, image or whatever) of which you know the downloadsize on your server, make a AJAX GET request to fetch it and measure the amount of time between the start of the download and when it's finished.
This can be done with only Javascript code, but I would probably recommend you to include a timestamp when the payload was generated in order to calculate the latency.
Upload:
Make a AJAX POST request towards the server which you fill with junk data of a specific size, calculate the time it took to perform this POST.
List of things to keep in mind:
You will not be able to measure higher bandwith than your servers
Concurrent users will drain the bandwith, build a "limit" if this will be a problem
It is hard to build a service to measure bandwidth with precision, the one you linked to is presenting 5Mbit/s to me when my actual speed is around 30Mbit/s according to Bredbandskollen.
Other links on the subject
What's a good way to profile the connection speed of Web users?
Simple bandwidth / latency test to estimate a users experience
Note: I have not built any service like this, but it should do the trick.
I need to generate thousands of personalized letters from more than 800 workstations in diverse locations.
While we had only a dozen of customers per location, we used Crystal Reports over the web and its PDF/Word export function to produce multi-page document for print. With thousand of customers this just doen't work for the following reasons:
Browser unable to reliably download huge report
Server resourses insufficient to handle simultanious requests from several workstations
Even if Server and Client are Ok, you cannot start printing until whole reort is downloaded (i.e. for hours)
Workstations only have browsers and local high perfromance printers at the moment.
What is the solution for this problem? How should I do this?
In the past, we had a similar challenge. To solve it, we created a C# application which was installed at each location. It would communicate with the server via web services to get the data it needed to print.
Then we used the native printing functionality built into .NET for the actual printing. The downloads were quick, and it could handle high volume printing.
Since the question was clarified as:
Remote warehouse only has browser and printer. It connects to the server and enters information about the item that just arrived. In responce to that, printing of thousands of shipping labels should start, warehouse cannot wait until labels are printed elswhere and delivered to its location.
...the rational way to do this would be to make all the printers network printers. The workstations could share them using Windows networking, or pick your own favourite print server technology.
The web application needs to be told which printer needs the labels. It then prints to whichever print server is appropriate.
The alternative - download the document and print locally - is too user-driven to be appropriate in my opinion.
I am building an ASP.NET web application that will be deployed to a 4-node web farm.
My web application's farm is located in California.
Instead of a database for back-end data, I plan to use a set of web services served from a data center in New York.
I have a page /show-web-service-result.aspx that works like this:
1) User requests page /show-web-service-result.aspx?s=foo
2) Page's codebehind queries a web service that is hosted by the third party in New York.
3) When web service returns, the returned data is formatted and displayed to user in page response.
Does this architecture have potential scalability problems? Suppose I am getting hundreds of unique hits per second, e.g.
/show-web-service-result.aspx?s=foo1
/show-web-service-result.aspx?s=foo2
/show-web-service-result.aspx?s=foo3
etc...
Is it typical for web servers in a farm to be using web services for data instead of database? Any personal experience?
What change should I make to the architecture to improve scalability?
You have most definitely a scalability problem: the third-party web service. Unless you have a service-level agreement with that service (agreeing on the number of requests that you can submit per second), chances are real that you overload that service with your anticipated load. That you have four nodes yourself doesn't help you then.
So you should a) come up with an agreement with the third party, and b) test what the actual load is that they can take.
In addition, you need to make sure that your framework can use parallel connections for accessing the remote service. Suppose you have a round-trip time of 20ms from California to New York (which would be fairly good), you can not make more than 50 requests over a single TCP connection. Likewise, starting new TCP connections for every request will also kill performance, so you want pooling on these parallel connections.
I don't see a problem with this approach, we use it quite a bit where I work. However, here are some things to consider:
Is your page rendering going to be blocked while waiting for the web service to respond?
What if the response never comes, i.e. the service is down?
For the first problem I would look into using AJAX to update the page after you get a response back from the web service. You'll also want to consider how to handle the no response or timeout condition.
Finally, you should really think about how you could cache the web service data locally. For example if you are calling a stock quoting service then unless you have a real-time feed, there is no reason to call the web service with every request you get. Store the data locally for a period of time and return that until it becomes stale.
You may have scalability problems but most of these can be carefully engineered around.
I recommend you use ASP.NET's asynchronous tasks so that the web service is queued up, the thread is released while the request waits for the web service to respond, and then another thread picks up when the web service is done to finish off the request.
MSDN Magazine - Wicked Code - Asynchronous Pages in ASP.NET 2.0
Local caching is an absolute must. The fewer times you have to go from California to New York, the better. You might want to look into Microsoft's Velocity (although that's still in CTP) or NCache, or another distributed cache, so that each of your 4 web servers don't all have to make and cache the same data from the web service - once one server gets it, it should be available to all.
Microsoft Project Code Named "Velocity"
NCache
Other things that can go wrong that you should engineer around:
The web service is down (obviously) and data falls out of cache, and you can't get it back. Try to make it so that the data is not actually dropped from cache until you're sure you have an update available. Then the only risk is if the service is down and your application pool is reset, so don't reset it as a first-line troubleshooting maneuver!
There are two different timeouts on web requests, a connect and an overall timeout. Make sure both are set extremely low and you handle both of them timing out. If the service's DNS goes down, this can look like quite a different failure.
Watch perfmon for ASP.NET Queued Requests. This number will rise rapidly if the service goes down and you're not covering it properly.
Research and adjust ASP.NET performance registry settings so you have a highly optimized ASP.NET thread pool. I don't remember the specifics, but I seem to remember that there's a limit on IO Completion Ports and something else of that nature that are absurdly low for the powerful hardware I'm assuming you have on hand.
the trendy answer is REST. Any GET request can be HTTP Response cached (with lots of options on how that is configured) and it will be cached by the internet itself (your ISP, essentially).
Your project has an architecture that reflects they direction that Microsoft and many others in the SOA world want to take us. That said, many people try to avoid this type of real-time risk introduced by the web service.
Your system will have a huge dependency on the web service working in an efficient manner. If it doesn't work, or is slow, people will just see that your page isn't working properly.
At the very least, I would get a web stress tool and performance test your web service to at least the traffic levels you expect to get at peaks, and likely beyond this. When does it break (if ever?), when does it start to slow down? These are good metrics to know.
Other options to look at: perhaps you can get daily batches of data from the web service to a local database and hit the database for your web site. Then, if for some reason the web service is down or slow, you could use the most recently obtained data (if this is feasible for your data).
Overall, it should be doable, but you want to understand and measure the risks, and explore any potential options to minimize those risks.
It's fine. There are some scalability issues. Primarily, with the number of calls you are allowed to make to the external web service per second. Some web services (Yahoo shopping for example) limit how often you can call their service and will lock out your account if you call too often. If you have a large farm and lots of traffic, you might have to throttle your requests.
Also, it's typical in these situations to use an interstitial page that forks off a worker thread to go and do the web service call and redirects to the results page when the call returns. (Think a travel site when you do search, you get an interstitial page while they call out to an external source for the flight data and then you get redirected to a results page when the call completes). This may be unnecessary if your web service call returns quickly.
I recommend you be certain to use WCF, and not the legacy ASMX web services technology as the client. Use "Add Service Reference" instead of "Add Web Reference".
One other issue you need to consider, depending on the type of application and/or data you're pulling down: security.
Specifically, I'm referring to authentication and authorization, both of your end users, and the web application itself. Where are these things handled? All in the web app? by the WS? Or maybe the front-end app is authenticating the users, and flowing the user's identity to the back end WS, allowing that to verify that the user is allowed? How do you verify this? Since many other responders here mention a local data cache on the front end app (an EXCELLENT idea, BTW), this gets even MORE complicated: do you cache data that is allowed to userA, but not for userB? if so, how do you verify that userB cannot access data from the cache? What if the authorization is checked by the WS, how do you cache the permissions then?
On the other hand, how are you verifying that only your web app is allowed to access the WS (and an attacker doesn't directly access your WS data over the Internet, for instance)? For that matter, how do you ensure that your web app contacts the CORRECT WS server, and not a bogus one? And of course I assume that all the connection to the WS is only over TLS/SSL... (but of course also programmatically verify the cert applies to the accessed server...)
In short, its complicated, and many elements to consider here.... but it is NOT insurmountable.
(as far as input validation goes, that's actually NOT an issue, since this should be done by BOTH the front end app AND the back end WS...)
Another aspect here, as mentioned by #Martin, is the need for an SLA on whatever provider/hosting service you have for the NY WS, not just for performance, but also to cover availability. I.e. what happens if the server is inaccessible how quickly they commit to getting it back up, what happens if its down for extended periods of time, etc. That's the only way to legitimately transfer the risk of your availability being controlled by an externality.