I want to differentiate between whether the traffic coming to DRAM is coming from page table walker (that is due to address translation, after it misses in the TLB), or it is other traffic (not coming from page table walker). Can I use Gem5 to do that?
A memory controller in gem5 can differentiate among different masters via the Request::masterId. A master will ask for a unique masterId at construction time, by using the System::getMasterId method.
Every time it will allocate a new packet/request, it will tag it with its own masterId.
So if you want to add a hook in the dram memory controller to perform some action if the request is coming from a specific master, you can easilly do that.
If you are just interested on high level stats, like the bandwidth utilization of a master, the DRAM controller should print that in the stat file (It would print average bw/latency per master).
Having said that I don't know which architecture/table-walker you are using.
The ARM table walker has its own masterId (even though its initialization is a little bit convoluted, I should fix this), but maybe that's not the case for the architecture you are using.
Anyway it should be trivial to modify the HTW to ask the System for a unique masterId token.
Related
I'm trying to create a trigger in zabbix which will show me a problem and alert me on my email whenever an interface in a cisco switch (with snmpv2) crosses 80% of it's bandwidth (100 mbps or 1000 mbps) without hardcoding anything, I tried using this trigger expression:
{/switch name:net.if.out[ifHCOutOctets./switch interface].min(10)}>80000000
I would like to know how can I write this trigger expression which works fine without applying it to every single interface item in every switch. I think that maybe macros could help in these situations but found no explanation or any guide about how to use them or how to use low level discovery which maybe have a part at the solution for my need.
Thanks in advance.
You are correct, you want to use Low Level Discovery to do this, as well as to discover all your interfaces. Low level discovery at a high level consists of two things. 1) you have to tell zabbix how to go discovery a bunch of dynamic things and asssign a LLD macro to them, that is done a the high level of the Discovery rule. 2) you have to tell zabbix what item protototypes, trigger protorypes, etc to dynamically create as actual items and triggers, every time the discovery rule runs.
Take a look at the Arista SNMPv2 template included with zabbix as an example. There are a number of Discovery Rules included in that template, one of which is the Network Interfaces discovery rule. Within the network interfaces discovery rule, zabbix basically does an snmp walk and gets a list of all the interfaces and assigns LLD(Low level discovery macros) for each interfaces such as #IFINDEX, #IFSTATUS, etc. The discovery Rule, like all LLD rules, takes the output of the "Network Interfaces" discovery rule, and uses them to dynamically create actual items on each host the template applied to.
The next part of this to understand is the prototypes. Once zabbbix finds all the network interfaces, your question should be, how do i get it to create new Items in my host for each interface it finds, and how do i get it to create triggers for each interface it finds dynamically, automatically and without user intervention. The answer is protoyptes. Prototypes are child elements of a Low Level Discovery. They are what actually creates the new items and triggers for every thing it discovered.
Take a look here for some examples and docs on low level discovery rules.
https://www.zabbix.com/documentation/4.2/manual/discovery/low_level_discovery#trigger_prototypes
Zabbix can create LLD rules via numerous discovery methods including SNMPv#, which is all capable of being configured in the UI or api, and other customer discovery rules not included through the use of user parmaters, externel checks, etc.
If your make and model of switch is already known to zabbix, a template in "Templates/Network Devices", at least i think thats the path, will exist just like the arista and juniper ones.
You can create custom low level discovery rules as well for non snmp stuff. basically you write a script that will go find the things you want to dynamically add to zabbix and your script needs to return a valid json output with #macronames and values you want added. For example a custom file system discovery rules, which shouldn't be needed because there already included if your using the agent, would produce lines like the ones shown in this example in the official docs.
https://www.zabbix.com/documentation/4.2/manual/discovery/low_level_discovery#creating_custom_lld_rules
In short, check to see if a template exists for your switch already and a discovery rule with the item prototypes to discover things the way you want them already. LLD basically allows zabbix too walk a dynamic data structure of any source, as long as that data structure has a definition known to zabbix, and you tell it what keys and values in the JSON you want to create as items, triggers, etc.
You should leverage the low level discovery feature of Zabbix.
After a standard setup you should have a "Template Module Interfaces SNMPv2", which is used by some other templates as a standard for interface discovery.
Within the template you will find a "Network Interfaces Discovery" discovery rule that runs every hour to:
query the target device for an interface list
create N items for each interface (bits in, bits out, speed, type etc), defined as Item Prototypes
create some triggers, defined as Trigger Prototypes
The speed item is the negotiated interface speed (ie: 1000 for a gigabit switch port), which is your 100% limit.
You can add some definitions to this template to get the alert you need:
Create a calculated item prototype and set its formula to
100*(currentOutputBits/speed)
Create a template macro to define your alert threshold, ie {$INTERFACE_OUTPUT_THRESHOLD}, and set it to 80
Create a trigger prototype that matches when the calculated item is
greater than {$INTERFACE_OUTPUT_THRESHOLD} for N minutes
Optionally, do the same for the currentInputBits item
This setup will create 1 additional item and 1 additional trigger for each physical or logical interface found on the target device: it makes sense to unflag the "create enable" option on the trigger prototype and enable it only on specific ports of specific devices.
The threshold macro can be changed at template level, affecting any device linked to it, or at host level for a custom threshold.
Thanks for your replies, I actually checked again the Template Module Interfaces SNMPv2 and saw that there is a prototype trigger that solves my question.
for anyone who wants to do the same with a switch:
Add to your switch (host) the "Template Module Interfaces SNMPv2" template.
Change the IF_UTIL_MAX macro to whatever value you want it to be, the default is 90 (this is the macro which is responsible for the percentage of the bandwidth which will trigger a problem, for example, if you change it to 60, when a host bandwidth utilization averages more than 60% in any interface for 15 minutes, a problem will be added to the problems tab or to the dashboard).
If the time of 15 minutes isn't right for you, you can change it by going to: configuration -> templates -> Template Module Interfaces SNMPv2 -> Discovery rules -> Network Interfaces Discovery -> Trigger prototypes -> search for a trigger name which contains high bandwidth usage -> in the problem expression and the recovery expression find the .avg() function and change the value inside it to whatever value is right for you, for an example: 15m = 15 minutes, 1s = 1 second etc...
I actually recommend to clone the trigger prototype, change it and then disable the built in one instead of just changing the time inside it for easily debugging errors in the long run. So you can change the name of the trigger prototype and then clone it by pressing clone in the bottom left corner of the screen, then change it's name and settings for whatever suits you the best.
Press add in the bottom left corner of the screen and if you took my advice you also need to click on the green "Yes" link of the built in trigger in the trigger prototypes table for it to disable.
You can also go ahead and try the other answers on this thread, I'm seriously thankful for them but I don't have enough time to check if those work since I already figured it out after reading Simone Zabberoni's and helllordkb's answers and checking the built in low level discovery in the "Template Module Interfaces SNMPv2" template.
Currently I use the AcknoledgingMessageListener to implement a Kafka consumer using spring-Kafka. This implementation helps me listen on a specific topic and process messages with a manual ack.
I now need to build the following capability:
Let us assume that for an some environmental exception or some entry of bad data via this topic, I need to replay data on a topic from and to a specific offset. This would be a manual trigger (mostly via the execution of a Java class).
It would be ideal if I can retrieve the messages between those offsets and feed it is a replay topic so that a new consumer can process those messages thus keeping the offsets intact on the original topic.
CosumerSeekAware interface - if this is the answer how can I trigger this externally? Via let say a mvn -Dexec. I am not sure if this is even possible
Also let say that I have an crash time stamp with me, is it possible to introspect the topic to find the offset corresponding to the crash so that I can replay from that offset?
Can I find offsets corresponding to some specific data so that I can replay those specific offsets?
All of these requirements are towards building a resilience layer around our Kafka capabilities. I need all of these to be managed by a separate executable class that can be triggered manually providing the relevant data (like time stamps etc). This class should determine offsets and then seek to that offset, retrieve the messages corresponding to those offsets and post them to a separate topic. Can someone please point me in the right direction? I’m afraid I’m going around in circles.
so that a new consumer can process those messages thus keeping the offsets intact on the original topic.
Just create a new listener container with a different group id (new consumer) and use a ConsumerAwareRebalanceListener (or ConsumerSeekAware) to perform the seeks when the partitions are assigned.
Here is a sample CARL that seeks all assigned topics based on a timestamp.
You will need some mechanism to know when the new consumer should stop consuming (at which time you can stop() the new container). Maybe set max.poll.records=1 on the new consumer so he doesn't prefetch past the failure point.
I am not sure what you mean by #3.
I would like to have pre-emption calls in Asterisk. I think there is no Asterisk support for this feature so i'm trying to implement it following a simliar algorithm like the one showed in this thread: Asterisk - Pre-emption calls
So I'm having problems in this step:
check if B in call with lower priority caller( ASTDB or REALTIME or fastagi script).
I know how to check if B is in a call using for example DEVICE_STATE(device) cmd, but i can't achieve to know who is the other caller in order to see his priority.
So, How can I know if one users is in a call and who is the other caller inside this call?
Thanks a lot.
You can read variables of any channel using
SHARED(varname[,channel])
-= Info about function 'SHARED' =-
[Synopsis]
Gets or sets the shared variable specified.
[Description]
Implements a shared variable area, in which you may share variables between
channels.
The variables used in this space are separate from the general namespace
of the channel and thus ${SHARED(foo)} and ${foo} represent two completely
different variables, despite sharing the same name.
Finally, realize that there is an inherent race between channels operating
at the same time, fiddling with each others' internal variables, which is
why this special variable namespace exists; it is to remind you that variables
in the SHARED namespace may change at any time, without warning. You should
therefore take special care to ensure that when using the SHARED namespace,
you retrieve the variable and store it in a regular channel variable before
using it in a set of calculations (or you might be surprised by the
result).
Sure you have set variables first.
You can set in variables or in ASTDB name of current speaking channel using in-call macro
General complexity of any solution like you want is above average, need person with at least 1-2 year of extensive experience with *.
I want to implement some file io with the routines provided by MPI (in particular Open MPI).
Due to possible limitations of the environment, I wondered, if it is possible to limit the nodes, which are responsible for IO, so that all other nodes are required to perform a hidden mpi_send to this group of processes, to actually write the data. This would be nice in cases, where e.g. the master node is placed on a node with high-performance filesystem and the other nodes have only access to a low-performance filesystem, where the binaries are stored.
Actually, I already found some information, which might be helpful, but I couldn't find further information, how to actually implement these things:
1: There is an info key MPI_IO belonging to the communicator, which tells which ranks provide standard-conforming IO-routines. As this is listed as an environmental inquiry, I don't see, where I could modify this.
2: There is an info key io_nodes_list which seems to belong to file-related info-objects. Unfortunately, the possible values for this key are not documented and Open MPI doesn't seem to implement them in any way. Actually, I can't even get the filename from the info-object which is returned by mpi_file_get_info...
As a workaround, I could imagine two things: On the one hand, I could perform the IO with standard Fortran routines, or on the other hand, create a new communicator, which is responsible for IO. But in both cases, the processes, which are responsible for IO have to check for possible IO from the other processes to perform manual communication and file interaction.
Is there a nice and automatic way to restrict the IO to certain nodes? If yes, how could I implement this?
You explicitly asked about OpenMPI, but there are two MPI-IO implementations in OpenMPI. The old workhorse is ROMIO, the MPI-IO implementation shared among just about every MPI implementation. OpenMPI also has OMPIO, but I don't know a whole lot about tuning that one.
Next, if you want things to happen automatically for you, you'll have to use collective i/o. The independent I/O routines cannot send a message to anyone else -- they are independent and there's no way to know if the other side will be listening.
With those preliminaries out of the way...
You are asking about "i/o aggregaton". There is a bit of information here in the context of another optimization called "deferred open" (and which OMPIO calls Lazy Open)
https://press3.mcs.anl.gov/romio/2003/08/05/deferred-open/
In short, you can definitely say "only these N processes should do I/O", and then the collective I/O library will exchange data and make sure that happens. The optimization was developed some 15-odd years ago for just the situation you proposed: some nodes being better connected to storage than others (as was the case on the old ASCI Red machine, to give you a sense for how old this optimization is...)
I don't know where you got io_nodes_list. You probably want to use the MPI-IO info keys cb_config_list and cb_nodes
So, you've got a cluster with master1, master2, master3, and compute1, compute2, compute3 (or whatever the hostnames actually are). You can do something like this (in c, sorry. I'm not proficient in Fortran):
MPI_Info info;
MPI_File fh;
MPI_Info_create(&info);
MPI_Info_set(info, "cb_config_list", "master1:1,master2:1,master3:1");
MPI_File_open(MPI_COMM_WORLD, filename, MPI_MODE_CREATE|MPI_MODE_WRONLY, info, &fh)
With these hints, MPI_File_write_all will aggregate all the I/O through the MPI processes on master1, master2, and master3. ROMIO won't blow up your memory because it will chunk up the I/O into a smaller working set (specified with the "cb_buffer_size" hint: cranking this up, if you have the memory, is a good way to get better performance).
There is a ton of information about the hints you can set in the ROMIO users guide:
http://www.mcs.anl.gov/research/projects/romio/doc/users-guide/node6.html
I have a situation where a main orchestration is responsible for processing a convoy of messages. These messages belong to a set of customers, the orchestration will read the messages as they come in, and for each new customer id it finds, it will spin up a new orchestration that is responsible for processing the messages of a particular customer. I have to preserve the order of messages as they come in, so the newly created orchestrations should process the message it has and wait for additional messages from the main orchestration.
Tried different ways to tackle this, but was not able to successfuly implement it.
I would like to hear your opinions on how this could be done.
Thanks.
It sounds like what you want is a set of nested convoys. While it might be possible to get that working, it's going to... well, hurt. In particular, my first worry would be maintenance: any changes to the process would be a pain in the neck to make, and, much worse, deployment would really, really suck.
Personally, I would really try to find an alternative way to implement this and avoid the convoys if possible, but that would depend a lot on your specific scenario.
A few questions, if you don't mind:
What are your ordering requirements? For example, do you only need ordered processing for each customer on a single incoming batch, or across batches? If the latter, could you make do without the master orchestration and just force a single convoy'd instance per customer? Still not great, but would likely simplify things a lot.
What are you failure requirements with respect to ordering? Should it completely stop processing? Save message and keep going? What about retries?
Is ordering based purely on the arrival time of the message? Is there anything in the message that you could use to force ordering internally instead of relying purely on the arrival time?
What does the processing of the individual messages do? Is the ordering requirement only to ensure that certain preconditions are met when a specific message is processed (for example, messages represent some tree structure that requires parents are processed before children).
I don't think you need a master orchestration to start up the sub-orchestrations. I am assumin you are not talking about the master orchestration implmenting a convoy pattern. So, if that's the case, here's what I might do.
There is a brief example here on how to implment a singleton orchestration. This example shows you how to setup an orchestration that will only ever exist once. All the messages going to it will be lined up in order of receipt and processed one at a time. Your example differs in that you want to have this done by customer ID. This is pretty simple. Promote the customer ID in the inbound message and add it to the correlation type. Now, there will only ever be one instance of the orchestration per customer.
The problem with singletons is this. You have to kill them at some point or they will live forever as dehydrated orchestrations. So, you need to have them end. You can do this if there is a way for the last message for a given customer to signal the orchestration that it's time to die through an attribute or such. If this is not possible, then you need to set a timer. If no messags are received in x seconds, terminate the orch. This is all easy to do, but it can introduce Zombies. Zombies occur when that orchestration is in the process of being shut down when another message for that customer comes in. this can usually be solved by tweeking the time to wait. Regardless, it will cause the occasional Zombie.
A note fromt he field. We've done this and it's really not a great long term solution. We were receiving customer info updates and we had to ensure ordered processing. We did this singleton approach and it's been problematic from the Zombie issue and the exeption issue. If the Singleton orchestration throws an exception, it will block the processing for a all future messages for that customer. So - handle every single possible exception. The real solution would have been to have the far end system check the time stamps from the update messages and discard ones that were older than the last update. We wanted to go this way, but the receiving system didn't want to do this extra work.