I have an ASP.NET page that gets the status of another page. If the string "serverx" appears on that page then that means it is up. I want to cache the results of responseBody for 5 minutes to not over load the requests to this server if someone spins up threads trying to abuse it.
string StatUrl = "https://www.server/staus";
try
{
string responseBody = await client.GetStringAsync(StatUrl);
if (responseBody.Contains("serverx"))
{
EcholinkStatus = "Online";
}
else
{
EcholinkStatus = "Offline";
}
}
catch (HttpRequestException e)
{
}
What is the simple and efficient way to only do one request per 5 minutes? Memory cache and is there an easy example to follow? Microsoft don't seem to make this simple from what I can find.
Related
Is it possible to execute a query asynchronously in hive server?
For eg, How can I /Is it possible to do something like this from the client-
QueryHandle handle = executeAsyncQuery(hiveQuery);
Status status = handle.checkStatus();
if(status.isCompleted()) {
QueryResult result = handle.fetchResult();
}
I also had a look at How do I make an async call to Hive in Java?. But did not help. The answers were mostly around the thrift clients taking a callback argument.
Any help would be appreciated. Thanks!
[EDIT 1]
I went through the HiveConnection.java in hive-jdbc. hive-jdbc by default uses the async thrift APIs. Hence it submits a query and polls for result sets (look at HiveStatement.java). Now i am able to write a piece of code which is purely non blocking. But the problem is as soon as the client disconnect the foot print about the query is lost.
Client 1
final TCLIService.Client client = new TCLIService.Client(createBinaryTransport(host, port, loginTimeout, sessConf, false)); // from HiveConnection.java
TSessionHandle sessionHandle = openSession(client) // from HiveConnection.java
TExecuteStatementReq execReq = new TExecuteStatementReq(sessionHandle, sql);
execReq.setRunAsync(true);
execReq.setConfOverlay(sessConf);
final TGetOperationStatusReq handle = client.ExecuteStatement(execReq)
writeHandleToFile("~/handle", handle)
Client 2
final TGetOperationStatusReq handle = readHandleFromFile("~/handle")
final TCLIService.Client client = new TCLIService.Client(createBinaryTransport(host, port, loginTimeout, sessConf, false));
while (true) {
System.out.println(client.GetOperationStatus(handle).getOperationState());
Thread.sleep(1000);
}
Client 2 keeps printing FINISHED_STATE as long as Client 1 is alive. But if client 1 process completes or gets killed, client 2 starts printing null which means hiveserver2 is cleaning up the resources as soon as a client disconnects.
Is it possible to configure hiveserver2 to configure this clean up process based on time or something?
Thanks!
Did some research and figured out that this happens only with binary transport (tcp)
#Override
public void deleteContext(ServerContext serverContext,
TProtocol input, TProtocol output) {
Metrics metrics = MetricsFactory.getInstance();
if (metrics != null) {
try {
metrics.decrementCounter(MetricsConstant.OPEN_CONNECTIONS);
} catch (Exception e) {
LOG.warn("Error Reporting JDO operation to Metrics system", e);
}
}
ThriftCLIServerContext context = (ThriftCLIServerContext) serverContext;
SessionHandle sessionHandle = context.getSessionHandle();
if (sessionHandle != null) {
LOG.info("Session disconnected without closing properly, close it now");
try {
cliService.closeSession(sessionHandle);
} catch (HiveSQLException e) {
LOG.warn("Failed to close session: " + e, e);
}
}
}
The above stub (from ThriftBinaryCLIService) gets executed through this piece of code from TThreadPoolServer which is used by ThriftBinaryCLIService.
eventHandler.deleteContext(connectionContext, inputProtocol,
outputProtocol);
Apparently http transport (ThriftHttpCLIService) has a different strategy of cleaning up operation handles (not greedy like tcp)
Will check with hive community on this to understand a bit more and see if there is an issue addressing this already.
I am using this code to invalidate CloudFront Files using ASP.NET AWS SDK.
public bool InvalidateFiles(string[] arrayofpaths)
{
try
{
var client = AWSClientFactory.CreateAmazonCloudFrontClient(MY_AWS_ACCESS_KEY_ID, MY_AWS_SECRET_KEY);
CreateInvalidationResponse r = client.CreateInvalidation(new CreateInvalidationRequest
{
DistributionId = ConfigurationManager.AppSettings["DistributionId"],
InvalidationBatch = new InvalidationBatch
{
Paths = new Paths
{
Quantity = arrayofpaths.Length,
Items = arrayofpaths.ToList()
},
CallerReference = DateTime.Now.Ticks.ToString()
}
});
}
catch
{
return false;
}
return true;
}
I supplied a path like "/images/1.jpg" but it seems that the invalidation doesn't take place. I've waited half and hour and used hard refresh and cleared the cache and the image is the same. I've changed the image to look different so I can notice the difference if invalidation has occurred.
In the console I see that the invalidation took place, because I have the ID and in the status I see 'completed', but again, nothing changed in the image.
My question is how can I check that the invalidation takes place and make it work, it seems that I am missing something.
Also, is there an option to create is asynchronous, so I won't need to answer from Amazon.
Thanks.
I have a requirement that a certain email distribution list should be notified every so often (still to be determined) about user accounts that are nearing expiration.
I'm wondering the best way to achieve this, I know its generally a bad idea to spawn another thread within asp.net to handle this type of thing, so I'm thinking maybe a simple service is the way to go but for something so small this seems like it might be slightly overkill.
Ideally I'd like something that doesnt require much babysitting (eg. checking service is running).
I have also suggested having a page in the site with this type of information but it is likely that it could be a few days before this is checked. We also cannot let users extend their own expiration date.
Are there any other viable options.
The best suitable method to work on it according to is
create a application which will select list of all users whose account expiry date is nearby (eg. 10 days from today) as per your requirement.
This application will be scheduled as an daily execution (you will create an exe with log file to display errors raised and total number of emails sent in one execution.)
This application will fetch all the records based on criteria and send the emails to all yours using the basic HTML template. and once the email is sent, you will update a column (notificationFlag) in your database as 1 if you have sent is once in last 10 days. else by default it will be 0
you can schedule the exe by the end of the day at 12:10 am (just incase your database server and webserver is not matching in time) every day. .
This is something I've done which is similar to Prescott's comment on your answer.
I have a website with an administrative page that reports on a bunch of expiration dates.
This page also accepts a QueryString parameter SEND_EMAILS, so anytime an administrative user of the site passes the QueryString parameter SEND_EMAILS=true a bunch of emails go out to all the users that are expiring.
Then I just added a windows scheduled task to run daily and load the page with the SEND_EMAILS=true parameter.
This was the simple code I used to issue the webrequest from the console in the scheduled task:
namespace CmdLoadWebsite
{
class Program
{
static void Main(string[] args)
{
string url = "http://default/site/";
if (args.Length > 0)
{
url = args[0];
}
Console.WriteLine(GetWebResult(url));
}
public static string GetWebResult(string url)
{
byte[] buff = new byte[8192];
StringBuilder sb = new StringBuilder();
HttpWebRequest request = (HttpWebRequest) WebRequest.Create(url);
HttpWebResponse response = (HttpWebResponse) request.GetResponse();
Stream webStream = response.GetResponseStream();
int count = 0;
string webString;
do
{
count = webStream.Read(buff, 0, buff.Length);
if (count != 0)
{
webString = Encoding.ASCII.GetString(buff, 0, count);
sb.Append(webString);
}
}
while (count > 0);
return(sb.ToString());
}
}
}
We are trying to load and resume workflows which have a delay. I have seen the Microsoft sample of Absolute Delay for this using store.WaitForEvents and LoadRunnableInstance to load the workflow. However here the workflow is already known.
In our case we want to have an event waiting for the store.WaitForEvents after every say 5 seconds to check if there is a runnable instance and if so only load and run that /those particular instances. Is there a way I could know which workflow instance is ready.
We are maintaing the workflow id and the xaml associated to it in our database, so if we could know the workflow instance id we could get the xaml mapped to it, create the workflow and then do a LOadRunnableInstance on it.
Any help would be greatly appreciated.
Microsoft sample (Absolute Delay)
public void Run(){
wfHostTypeName = XName.Get("Version" + Guid.NewGuid().ToString(),
typeof(WorkflowWithDelay).FullName);
this.instanceStore = SetupSqlpersistenceStore();
this.instanceHandle =
CreateInstanceStoreOwnerHandle(instanceStore, wfHostTypeName);
WorkflowApplication wfApp = CreateWorkflowApp();
wfApp.Run();
while (true)
{
this.waitHandler.WaitOne();
if (completed)
{
break;
}
WaitForRunnableInstance(this.instanceHandle);
wfApp = CreateWorkflowApp();
try
{
wfApp.LoadRunnableInstance();
waitHandler.Reset();
wfApp.Run();
}
catch (InstanceNotReadyException)
{
Console.WriteLine("Handled expected InstanceNotReadyException, retrying...");
}
}
Console.WriteLine("workflow completed.");
}
public void WaitForRunnableInstance(InstanceHandle handle)
{
var events=instanceStore.WaitForEvents(handle, TimeSpan.MaxValue);
bool foundRunnable = false;
foreach (var persistenceEvent in events)
{
if (persistenceEvent.Equals(HasRunnableWorkflowEvent.Value))
{
foundRunnable = true;
break;
}
}
if (!foundRunnable) {
Console.WriteLine("no runnable instance");
}
}
Thanks
Anamika
I had a similar problem with durable delay activities and WorkflowApplicationHost. Ended up creating my own 'Delay' activity that worked essentially the same way as the one out of the box, (takes an arg that describes when to resume the workflow, and then bookmarks itself). Instead of saving delay info in the SqlInstanceStore though, my Delay Activity created a record in a seperate db. (similar to the one you are using to track the Workflow Ids and Xaml). I then wrote a simple service that polled that DB for expired delays and initiated a resume of the necessary workflow.
Oh, and the Delay activity deleted it's record from that DB on bookmark resume.
HTH
I'd suggest having a separate SqlPersistenceStore for each workflow definition you're hosting.
What tools or techniques can I use to protect my ASP.NET web application from Denial Of Service attacks
For sure a hardware solution is the best option to prevent DOS attacks, but considering a situation in which you have no access to hardware config or IIS settings, this is definitely why a developer must have something handy to block or at least decrease dos attack effect.
The core concept of logic relies on a FIFO (First In First Out) collection such as Queue, but as it has some limitations I decided to create my own collection.
Without discussing more details this is the complete code I use:
public class AntiDosAttack
{
readonly static List<IpObject> items = new List<IpObject>();
public static void Monitor(int Capacity, int Seconds2Keep, int AllowedCount)
{
string ip = HttpContext.Current.Request.UserHostAddress;
if (ip == "")
return;
// This part to exclude some useful requesters
if(HttpContext.Current.Request.UserAgent != null && HttpContext.Current.Request.UserAgent == "Some good bots")
return;
// to remove old requests from collection
int index = -1;
for (int i = 0; i < items.Count; i++)
{
if ((DateTime.Now - items[i].Date).TotalSeconds > Seconds2Keep)
{
index = i;
break;
}
}
if (index > -1)
{
items.RemoveRange(index, items.Count - index);
}
// Add new IP
items.Insert(0, new IpObject(ip));
// Trim collection capacity to original size, I could not find a better reliable way
if (items.Count > Capacity)
{
items.RemoveAt(items.Count - 1);
}
// Count of currect IP in collection
int count = items.Count(t => t.IP == ip);
// Decide on block or bypass
if (count > AllowedCount)
{
// alert webmaster by email (optional)
ErrorReport.Report.ToWebmaster(new Exception("Blocked probable ongoing ddos attack"), "EvrinHost 24 / 7 Support - DDOS Block", "");
// create a response code 429 or whatever needed and end response
HttpContext.Current.Response.StatusCode = 429;
HttpContext.Current.Response.StatusDescription = "Too Many Requests, Slow down Cowboy!";
HttpContext.Current.Response.Write("Too Many Requests");
HttpContext.Current.Response.Flush(); // Sends all currently buffered output to the client.
HttpContext.Current.Response.SuppressContent = true; // Gets or sets a value indicating whether to send HTTP content to the client.
HttpContext.Current.ApplicationInstance.CompleteRequest(); // Causes ASP.NET to bypass all events and filtering in the HTTP pipeline chain of execution and directly execute the EndRequest event.
}
}
internal class IpObject
{
public IpObject(string ip)
{
IP = ip;
Date = DateTime.Now;
}
public string IP { get; set; }
public DateTime Date { get; set; }
}
}
The internal class is designed to keep the date of request.
Naturally DOS Attack requests create new sessions on each request while human requests on a website contain multiple requests packed in one session, so the method can be called in Session_Start.
usage:
protected void Session_Start(object sender, EventArgs e)
{
// numbers can be tuned for different purposes, this one is for a website with low requests
// this means: prevent a request if exceeds 10 out of total 30 in 2 seconds
AntiDosAttack.Monitor(30, 2, 10);
}
for a heavy request website you may change seconds to milliseconds but consider the extra load caused by this code.
I am not aware if there is a better solution to block intentional attacks on website, so I appreciate any comment and suggestion to improve the code. By then I consider this as a best practice to prevent DOS attacks on ASP.NET websites programmatically.
Try the Dynamic IP Restriction extension http://www.iis.net/download/dynamiciprestrictions
Not a perfect solution, but helps raise the bar =)
It's a broad area, so if you can be more specific about your application, or the level of threat you're trying to protect against, I'm sure more people can help you.
However, off the bat, you can go for a combination of a caching solution such as Squid: http://www.blyon.com/using-squid-proxy-to-fight-ddos/, Dynamic IP Restriction (as explained by Jim) and if you have the infrastructure, an active-passive failover setup, where your passive machine serves placeholder content which doesnt hit your database / any other machines. This is last-defence, so that you minimise the time a DDOS might bring your entire site offline for.