Find missing URL routes using the command-line - unix

I'm trying to automate a check for missing routes a Play! web application.
The routing table is in a file in the following format:
GET /home Home.index
GET /shop Shop.index
I've already managed to use my command line-fu to crawl through my code and make a list of all the actions that should be present in the file. This list is in the following format:
Home.index
Shop.index
Contact.index
About.index
Now I'd like to pipe the output of this text into another command that checks if each line is present in the route file. I'm not sure how to proceed though.
The result should be something like this:
Contact.index
About.index
Does someone have a helpful suggestion on how I can accomplish this?

try this line:
awk 'NR==FNR{a[$NF];next}!($0 in a)' routes.txt list.txt
EDIT
if you want the above line to accept list from stdin:
cat list.txt|awk 'NR==FNR{a[$NF];next}!($0 in a)' routes.txt -
replace cat list.txt with your magic command

Related

Can I use conditional "or" statements in selecting files to download with wget

I'm trying to download a bunch of files via ftp with wget. I could do this manually for each of the variables that I am interested in, or I was wondering if I could specify these in an "or" type conditional statement in the filepath name.
For example, I would like to download all files that contain the strings "NRRS412", "NRRS443", "NRRS490", etc. I had planned to do individual calls to wget for each of these, like this:
wget -r -A "L3m*NRRS412*.nc" ftp://username:password#ftp.address
I cannot simply use "L3m*NRRS*.nc", as there are other "NRRS" strings that I don't want.
Is there a way to download all of my target strings in a single call to wget?
Thanks for any help
OK, I figured out the solution, which is to create several possible strings separated by commas:
wget -r -A "L3m*NRRS412*.nc, L3m*NRRS43*.nc, L3m*NRRS490*.nc" ftp://username:password#ftp.address

Finding and Replacing a filename in a css File with Bash script

I want to write a bash script that takes a user input (which will be a filename) and replaces a path to a file inside a css file with that filename. For simplicity, the two files will be in the same folder and in the css code only the filename at the end of the path should be changed.
I thought of using regex to match any line of code that has a specific pattern and then change the end of it. I know about sed, but since the filename always changes I'm not sure how to solve this problem other than regex. I also thought of adding a variable in the css file that holds the filename as a value and then adding that variable at the end of the path, but I'm not sure then how to access that variable from a bash script.
Any recommendations on how to tackle this problem?
Thanks!
Edit Adding more Information:
Here is the line in the css file I want to edit. The part to be changed is the fileName.png at the end. Since it will change I thought of using a regex to "find" the correct spot in the css file.
background: #2c001e url(file:////usr/share/backgrounds/fileName.png/);
A regex matching only this line in this specific file is the following. It could probably be simplified, but I don't see a reason why since it should work too:)
(background)\:\s\#.{6}\s(url)\((file)\:\/{4}(usr)\/(share)\/backgrounds\/.+\.(png)\/\)\;
So, there are some ways to do that. You can check topic in links below. sed command is also good idea. But before executing it, you can build a new variable (or multiple variables) to use them in regex sed -e syntax.
Getting the last argument passed to a shell script
Maybe, if you will add some input and output examples, I could be more specific in this case.
To replace the input in the file at run-time you could use this line in a script
sed "s/stringToReplace/$1/g" templateFile >fileToUse
the $1 is referencing the 2nd bash script argument (the first being $0, the name of the invoking script). stringToReplace would be written in verbatim in the templateFile.
You could also use a script with two runtime arguments ($1, $2), and you would change the original contents of the fileToUse using the -i option. But this requires storage of the last file path to be used as argument $1.

How to handle a file having header in between the records after removing duplicates from the file

We have a file which has been processed by unix command for removing duplicates. After the de-duplication new file has the header in-between the records. Please help to solve this and thanks in advance for inputs.
Unix Command : Sort -u >
I would do something like this:
grep "headers" >output.txt
grep -v "headers" >>output.txt
The idea is the following: first take the headers and put them into output.txt, and afterwards take everything which is not a header and put it into that output file.
First you need to put the information in the output file (which means you need to create the output file, hence the single > character), secondly you need to append the information to the already existing output file (hence the double >> character).

How to generate translation file (.po, .xliff, .yml,...) from a Symfony2/Silex project?

Im going to build a Silex/Symfony2 project and I have been looking around for a method to generate XLIFF/PO/YAML translation files based on texts-to-be-translated inside the project but not found any instruction or documentation on it.
My question is: Is there an automated way to generate translation file(s) in specific format for a Symfony2/Silex project?
If yes, please tell me how to generate the file then update the translation after that.
If no, please tell me how to create translation file(s) then adding up more text for my project? I am looking for an editor desktop based or web-based instead of using normal editor such as Transifex, GetLocalization (but they dont have option to create a new file or add more text)
After a long time searching the internet, I found a good one:
https://github.com/schmittjoh/JMSTranslationBundle
I see you've found a converter, but to answer your first question about generating your initial translation file -
If you have Gettext installed on your system you could generate a PO file from your "texts-to-be-translated inside the project". The command line program xgettext will scan the source files looking for whatever function you're using.
Example:
To scan PHP files for instances of the trans method call as shown here you could use the following command -
find . -name "*.php" | xargs xgettext --language=PHP --keyword=trans --output=messages.pot
To your question about editors:
You could use any PO editor, such as POEdit, to manage your translations, but as you say you eventually need to convert the PO file to either an XLIFF or YAML language pack for Symfony.
I see you've already found a converter tool. You may also like to try the one I wrote for Loco. It supports PO to YAML, and PO to XLIFF
Workaround for busy people (UNIX)
You can run the following command in the Terminal:
$ grep -rEo --no-filename "'.+'\|\btrans\b" templates/ > output.txt
This will output the list of messages to translate:
'Please provide your email'|trans
'Phone'|trans
'Please provide your phone number'|trans
...
I mean almost.. But you can usually do some work from here...
Obviously you must tweak the command to your liking (transchoice, double-quotes instead of single...).
Not ideal but can help!
grep options
grep -R, -r, --recursive: Read all files under each directory, recursively this is equivalent to the -d recurse option.
grep -E, --extended-regexp: Interpret PATTERN as an extended regular expression.
grep -o, --only-matching: Show only the part of a matching line that matches PATTERN.
grep -h, --no-filename: Suppress the prefixing of filenames on output when multiple files are searched.
(source)

script to extract the details from xml

if have any xml file as below:
<soap env="abc" id="xyz">
<emp>acdf</emp>
<Workinstance name="ab" id="ab1">
<x>1</x>
<y>2</y>
</Workinstance>
<projectinstance name="cd" id="cd1">
<u>1</u>
<v>2</v>
</projectinstance>
</soap>
I want to extract the id field in workinstance using unix script
I tried grep but, it is retrieving the whole xml file.
Can someone help me how to get it?
You might want to consider something like XMLStarlet, which implements the XPath/XQuery specifications.
Parsing XML with regular expressions is essentially impossible even under the best of conditions, so the sooner you give up on trying to do this with grep, the better off you're likely to be.
XmlStarlet seems the tool I was looking for!
To do extract your tag, try to do the following:
cat your_file.xml | xmlstarlet sel -t -v 'soap/Workinstance/#id'
The "soap/Workinstance/#id" is an XPath expression that will get the id attribute inside Workinstance tag. By using "-v" flag, you ask xmlstarlet to print the extracted text to the standard output.
If you have Ruby
$ ruby -ne 'print $_.gsub(/.*id=\"|\".*$/,"" ) if /<Workinstance/' file
ab1

Resources