I am a unix addict, and many of the machines I use (at home and at work) are now quickly passing the 10,000 command mark. I like to keep all of the commands I issue readily available which is why I have set the upper limit to something like 100,000 entries, but it is becoming tedious to recall particular recent entries as I have to be writing !12524 in the shell to expand out that one.
Sure, I can use the shortcut ones to recall the last command or even the tenth-last command but its impossible to keep track of that so 90% of the time I am doing things like history | grep 'configure --prefix' (for reviewing how I configured something last, etc) and then using whatever history index that spits back.
Can I reverse it so that command #10000 corresponds to the command from ten thousand commands ago, and have command #1 be the last command?
Related
In R, I like to use reverse search (ctrl+r) to redo infrequent but complex commands without a script. Frequently, I will do so many other commands in between that the command history discards the old command. How can I change the default length of the command history?
This is platform and console specific. From the help for ?savehistory:
There are several history mechanisms available for the different R
consoles, which work in similar but not identical ways...
...
The history mechanism is controlled by two environment variables:
R_HISTSIZE controls the number of lines that are saved (default 512),
and R_HISTFILE sets the filename used for the loading/saving of
history if requested at the beginning/end of a session (but not the
default for these functions). There is no limit on the number of lines
of history retained during a session, so setting R_HISTSIZE to a large
value has no penalty unless a large file is actually generated.
So, in theory, you can read and set R_HISTSIZE with:
Sys.getenv("R_HISTSIZE")
Sys.setenv(R_HISTSIZE = new_number)
But, in practise, this may or may not have any effect.
See also ?Sys.setenv and ?EnvVar
Take a look at the help page for history(). This is apparently set by the environment variable R_HISTSIZE so you can set it for the session with Sys.setenv(R_HISTSIZE = XXX). I'm still digging to find where you change this default behavior for all R sessions, but presumably it will be related to .Startup or your R profile.
?history
"There are several history mechanisms available for the different R
consoles, which work in similar but not identical ways. "
Furthermore there may even be two history mechanism in the same device. I have .history files saved from the console and the Mac R GUI has its own separate system. You can increase the number of GUI managed history entries in the Preferences panel.
There is an incremental history package:
http://finzi.psych.upenn.edu/R/library/track/html/track.history.html
In ESS you can set comint-input-ring-size:
(add-hook 'inferior-ess-mode-hook
'(lambda()
(setq comint-input-ring-size 9999999)))
I need to obtain statistics about the network traffic of an mpls link between two sites. The main purpose of this is detect the 'top flooders' at the end of the day and at precise moments when the network is 'overloaded'.
At this time i have a sniffer with Ubuntu and i'm using wireshark to capture packets. The built-in statistics are awesome, but i can only use them with not bigger files than 150mb (it hungs for memory leaks with bigger files). So i use them for precise moments to detect in 'live mode' any instant flooder. But its impossible for me to leave wireshark capturing traffic all day long because of the hungs.
What tools are better suited to use them for these purposes? (detect any 'instant' flooder and take statistics of top talkers and top conversations between computers for the entire day)
Thank you.
Preliminary important note:
wireshark does not "hang for memory leaks with bigger files". The (very annoying) problem is that when opening a file, wireshark dissect it entirely from first to last packet before doing anything else and 1/ that can take a very very very long time and 2/ this imply that wireshark will have the entire file in memory e.g. the wireshark process will weight 1GB of memory for a 1GB trace (plus its own internal memory data of course), which may becomes a problem not only for wireshark but for the whole OS. Hence yes, it can become so unresponsive for so long that it looks like it's "hanged". Not a bug - rather a missing very complicated feature to dissect in "lazy" mode. The same goes with live capture, it dissect and put to relation everything (so that it knows and follow TCP dialog for instance) on the fly, and will hold the entire capture in memory. Which can quickly becomes quite heavy, both on memory and CPU.
And this will not the fixed implemented tomorrow, so now to your problem:
An option would be not to save to a file and latter process it, but doing it "live". You can do so using tshark (a terminal base version of wireshark) that will do the capture just like wireshark, and pipe its textual output to a dissecting/statistic analysis of your own.
https://www.wireshark.org/docs/man-pages/tshark.html
It has a -Y <displaY filter> option, so you should be able to use the MPLS filters from wireshark:
https://www.wireshark.org/docs/dfref/m/mpls.html
The -z <statistics> option will not be usable since it display the result after finishing reading the capture file, and you'll be piping live.
And tshark by default work in "one-pass analysis" mode, which of course limit a lot the analysis it can do, but alleviate the wireshark issue of "I want to dissect everything".[*]
So that would look like:
$ sudo tshark -i <your interface> -Y <your display filters> etc etc | your_parsing_and_statistical_tool
Of course, you'll have to write your own code for "your_parsing_and_statistical_tool". I'm not familiar with MPLS, nor know the statistics your interested in, but that may just be a couple of hours (or days) or Python coding? So if that's worth it for your job...
[*]:
tshark also have an option -2 to perform a two-pass analysis, but that would not work here since the first-pass must be completed first, which will never occur since your not reading a file but capture and analyse live.
I'm working on a graph containing about 50 million nodes and 40 million relationships.
I need to update every node.
I'm trying to set a new label to these nodes, but it's taking too long.
The label applies to all 50 million nodes, so the operation never ends.
After some research, i found out that Neo4j treats this operation as a single transaction (i don't know if optimistic or not), keeping the changes uncommitted, until the end (which will never happen in this fashion).
I'm currently using Neo4j 2.1.4, which has a feature called "USING PERIODIC COMMIT" (already present in earlier versions). Unfortunately, this feature is coupled to the "LOAD CSV" feature, and not available to every cypher command.
The cypher is quite simple:
match n set n:Person;
I decided to use a workaround, and make some sort of block update, as follows:
match n
where not n:Person
with n
limit 500000
set n:node;
It's ugly, but i couldn't come up with a better solution yet.
Here are some of my confs:
== neo4j.properties =========
neostore.nodestore.db.mapped_memory=250M
neostore.relationshipstore.db.mapped_memory=500M
neostore.propertystore.db.mapped_memory=900M
neostore.propertystore.db.strings.mapped_memory=1300M
neostore.propertystore.db.arrays.mapped_memory=1300M
keep_logical_logs=false
node_auto_indexing=true
node_keys_indexable=name_autocomplete,document
relationship_auto_indexing=true
relationship_keys_indexable=role
execution_guard_enabled=true
cache_type=weak
=============================
== neo4j-server.properties ==
org.neo4j.server.webserver.limit.executiontime=20000
org.neo4j.server.webserver.maxthreads=200
=============================
The hardware spec is:
RAM: 24GB
PROC: Intel(R) Xeon(R) X5650 # 2.67GHz, 32 cores
HDD1: 1.2TB
In this environment, each block update of 500000 nodes took from 200 to 400 seconds. I think this is because every node satisfies the query at the start, but as the updates take place, more nodes need to be scanned to find the unlabeled ones (but again, it's a hunch).
So what's the best course of action whenever an operation needs to touch every node in the graph?
Any help towards a better solution to this will be appreciated!
Thanks in advance.
The most performant way to achieve this is using the batch inserter API. You might use the following recipe:
take a look at http://localhost:7474/webadmin and note the "node count". In fact it's not the number of nodes it's more the highest node id in use - we'll need that later on.
make sure to cleanly shut down your graph database.
take a backup copy of your graph.db directory.
write a short piece of java/groovy/(whatever jvm language you prefer...) program that performs the following tasks
open your graph.db folder using the batch inserter api
in a loop from 0..<node count> (from step above) check if the node with given id exists, if so grab its current labels and amend the list by the new label and use setNodeLabels to write it back.
make sure you run shutdown with the batchinserter
start up your Neo4j instance again
ls reports the following:
# ls -ld /var/lib/puppet/state/
drwxr-xr-t 3 puppet puppet 4096 Jan 8 16:53 /var/lib/puppet/state/
What does the "t" mean for other? What tool reports the symbolic names for the permissions? ls has the --numeric-uid-gid option, but is there another one for permissions?
man ls is your friend:
t The sticky bit is set (mode 1000), and is searchable or executable.
(See chmod(1) or sticky(8).)
About sticky bit:
When set, it instructed the operating system to retain the text segment of the program in swap space after the process exited. This speeds up subsequent executions by allowing the kernel to make a single operation of moving the program from swap to real memory. Thus, frequently-used programs like editors would load noticeably faster.
The sticky bit is today incredibly confusing. It no longer pins the file into memory, making it actually "sticky", anymore. Sometimes it is called the "tacky" bit because it is represented as a 't' or 'T', but other folks still call it sticky. It only matters in modern linux and unix when applied to a directory, as far as I can tell. It means that people in groups who have the permission to delete a file still can't do it if the sticky bit is set on the directory. But it gets more confusing. It shows up in the last field, which is the execute/search field for "other" users, but acts on "group" users ("other" normal users can never delete files). The reason why it isn't shown in the "group" execute field is because that one changes to an "s" if the SETUID bit is set for group. I think. I am still researching that one.
So I have a website which has three spaces for pictures: Space A, Space B, and Space C.
In Space A, I have 20 images (Images A1-A20) which I would like to rotate daily.
In Space B, I have 20 images (Images B1-B20) which I would like to change every 13 days.
In Space C, I have 20 images (Images C1-C20) which I would like to change every 20 days.
Therefore, let's say for instance I wanted my site to start based on Jan 1, 2000.
Spaces A,B,&C would have the following images based on the date:
1/1 - A1,B1,C1
1/2 - A2,B1,C1
1/3 - A3,B1,C1
...
1/12 - A12,B1,C1
1/13 - A13,B1,C1
1/14 - A14,B2,C1
1/15 - A15,B2,C1
...
1/20 - A20,B2,C1
1/21 - A1,B2,C2
1/22 - A2,B2,C2
1/23 - A3,B2,C2
...
1/26 - A6,B2,C2
1/27 - A7,B3,C2
1/28 - A8,B3,C2
etc.
Therefore, I think I need a DateTime program that finds the number of elapsed days between the starting date (Jan 1, 2000), and the date listed on the computer, divides by the appropriate number (20,260,400), uses the remainder or fraction to determine which picture is appropriate and picks it to display, but haven't a clue how to write it / where to start.
Suggestions appreciated.
Thanks, NP
If you've been looking for a reason to learn php or perl or similar, this is it. If not, then try rent-a-coder or similar. There's a lot more to getting your first program working than just the code--you need to figure out how to get the software loaded and working correctly on your server.
The good news is that you seem to know your problem statement well. I'd go to a bookstore and look at the php or perl books until you see one that seems helpful, then go for it. And we here at SO stand ready to answer all questions from true seekers...
ps. For your problem, php would be a good starting bet if you're using a Linux web server.
Larry K is correct in that you have the problem pretty well defined and will need some sort of program to do this. PHP and Perl would be good choices, but there are ways in which you could do this with virtually any language: Python, C, Ruby, .... One technique would be to run a cron job (assuming Linux or another Unix variant) to re-write your HTML file every day (at midnight or thereabouts). Another would be to write a CGI script to determine the proper images when the page is requested. The PHP way (or Perl or Python if using mod_perl or mod_python) would embed (or pull in) the code directly in the web page and execute when the page is requested. I suggest you play around and pick what works best for you and your situation.
FWIW, if it were me, and I could do it any way I liked, I'd probably write a Python script to run as a cron job. Python just because that's currently my favorite language, and doing it by a cron job minimizes the processing required to do the job by doing it just once per day. However, some web hosting companies don't provide cron (or Python :-( ); if that were the case, I'd do it with PHP embedded in the web page, assuming I could use PHP (which seems ubiquitous with web hosting companies).