Ignore dags in subfolders using .airflowignore - airflow

I have the following dir structure:
.
├── project
│   ├── dag_1
│   │   ├── dag
│   │   │   ├── current
│   │   │   └── dag_1_v2.py
│   │   │   └── deprecated
│   │   │   └── dag_1_v1.py
│   │   └── sparkjobs
│   │   ├── current
│   │   └── deprecated
│   └── dag_2
│      ├── dag
│      │   ├── current
│      │   └── dag_2_v2.py
│      │   └── deprecated
│      │   └── dag_2_v1.py
│   └── sparkjobs
│   ├── current
│   └── deprecated
I want to ignore all deprecated folders, so I used .airflowignore to do that. When I place .airflowignore with */deprecated inside dag_1 or dag_2 folder, Airflow ignores the deprecated dag, like:
├── project
│   ├── dag_1
│   │   ├── .airflowignore
│   │   ├── dag
│   │   │   ├── current
│   │   │   └── dag_1_v2.py
│   │   │   └── deprecated
│   │   │   └── dag_1_v1.py
Considering this, I'll have to place a .airflowignore inside each dag folder. When I try to put onlny one .airflowignore using **/**/deprecated in the project folder the deprecated dags returns to Airflow, like:
├── project
│   ├── .airflowignore
│   ├── dag_1
│   │   ├── dag
│   │   │   ├── current
│   │   │   └── dag_1_v2.py
│   │   │   └── deprecated
│   │   │   └── dag_1_v1.py
My question is: How can I have only one .airflowignore in the project dir level to ignore all deprecated folders inside each dag_n/dag folder? Is this possible?

.airflowignore has same logic as .gitignore so what ever solution applies to .gitignore will also work here.
I believe what you are after is just
deprecated/
on the top level.
See also ignoring any 'bin' directory on a git project

Related

Node js Express app - css breaks when routing with more than one child path/parameter path

So if I try to load a page for example /patient_profile the css loads correctly, it's when I continue the path to /patient_profile/settings_patient_profile the css just breaks, both pages that I mentioned above are the same file, I just tried to test the routing to see if it works.
the route is managed like this:
route('/patient_profile/:id')
route('/patient_profile/settings_patient_profile/:id')
And I do land in the desired page, only the design elements don't work.
I read in another topic here that setting a static route with express to assets in my situation should fix the issue, but it didn't, I probably didn't understand how correctly use static path.
this directory tree:
.
├── docs
├── LICENSE
├── package.json
├── package-lock.json
├── README.md
├── src
│   │   ├── bootstrap
│   │   │   ├── css
│   │   │   │   └── bootstrap.min.css
│   │   │   └── js
│   │   │   └── bootstrap.min.js
│   │   ├── css
│   │   │   ├── FontAwesome.css
│   │   │   ├── font-awesome.min.css
│   │   │   ├── Footer-Basic-1.css
│   │   │   ├── Footer-Basic.css
│   │   │   ├── Login-Form-Clean.css
│   │   │   ├── Navigation-with-Button.css
│   │   │   ├── Registration-Form-with-Photo.css
│   │   │   └── style.css
│   │   ├── fonts
│   │   ├── img
│   │   │   ├── avatars
│   │   │   ├── dogs
│   │   └── js
│   │   ├── bs-init.js
│   │   ├── chart.min.js
│   │   ├── custom.js
│   │   ├── jquery.min.js
│   │   ├── smoothscroll.js
│   │   └── theme.js
│   ├── controllers
│   │   └── user_controller.js
│   ├── fonts
│   ├── img
│   ├── index.js
│   ├── js
│   ├── routes
│   │   ├── profile_route.js
│   │   └── route.js
│   └── views
│   ├── index.ejs
│   ├── partials
│   ├── patient_profile.ejs
│   ├── previous_appointments.ejs
├── test
│   └── array.spec.js
└── tree.txt
I ignored some files like fonts and pictures to make the tree more readable
index.js
const express = require('express')
const routes = require('./routes/route')
const appPort = process.env.PORT
const app = express()
//.env
require('dotenv').config({
path: path.join(__dirname, './.env')
})
//Setting up Handlebars
app.set('view engine', 'ejs')
app.set('views', __dirname + '/views')
app.use('/src', express.static(__dirname + '/src'))
app.use('/views', express.static(__dirname + '/views'))
app.use('/fonts', express.static(__dirname + '/fonts'))
app.use('/js', express.static(__dirname + '/js'))
app.use('/img', express.static(__dirname + '/img'))
app.use('/assets', express.static(__dirname + '/assets'))
app.use('/', routes)
//Creating a connection
app.listen(appPort, () => {
console.log(`App is running. serve at port: ${appPort}`)
console.log(`http://127.0.0.1:${appPort}`)
})
Again, sanitized programming related to DB etc.
my css referral in settings_patient_profile.ejs:
<link rel="stylesheet" href="../assets/bootstrap/css/bootstrap.min.css">
<link href='http://fonts.googleapis.com/css?family=BenchNine:300,400,700' rel='stylesheet' type='text/css'>
<link rel="stylesheet" href="../assets/fonts/fontawesome-all.min.css">
<link rel="stylesheet" href="../assets/fonts/font-awesome.min.css">
<link rel="stylesheet" href="../assets/fonts/fontawesome5-overrides.min.css">
For your CSS files to work reliably, no matter what the path of the parent page is, you need to use absolute URLs for the CSS links, not relative links. So, start the link with /, not with ../.
You don't show exactly where the target CSS files are in your server file system, but I would guess that you need to change this:
<link rel="stylesheet" href="../assets/fonts/fontawesome-all.min.css">
to something like this:
<link rel="stylesheet" href="/assets/fonts/fontawesome-all.min.css">
And, then use one express.static() middleware that serves everything in the assets hierarchy:
app.use('/assets', express.static(__dirname + '/assets/src'));
This particular recommendation is based on the view of your file system that shows all the assets are in /assets/src on your server disk.
Also, as long as you're willing to prefix all your static URLs with /assets, you should not be needing all of these as they can all be handled by the one previous express.static() middleware:
app.use('/src', express.static(__dirname + '/src'))
app.use('/views', express.static(__dirname + '/views'))
app.use('/fonts', express.static(__dirname + '/fonts'))
app.use('/js', express.static(__dirname + '/js'))
app.use('/img', express.static(__dirname + '/img'))

How do I change Bulma's default styling?

I've installed sass as instructed here http://versions.bulma.io/0.7.0/documentation/components/navbar/#variables. The guide says I need to include my change from $navbar-item-hover-background-color: $background to $navbar-item-hover-background-color: #28567d before bulma.sass gets imported.
I haven't seen a line with #import bulma.sass or #import bulma on it though.
the directory tree may be helpful:
.
├── bulma.sass
├── CHANGELOG.md
├── css
│   ├── bulma.css
│   └── bulma.css.map
├── LICENSE
├── package.json
├── README.md
└── sass
├── base
│   ├── _all.sass
│   ├── generic.sass
│   ├── helpers.sass
│   └── minireset.sass
├── components
│   ├── _all.sass
│   ├── breadcrumb.sass
│   ├── card.sass
│   ├── dropdown.sass
│   ├── level.sass
│   ├── media.sass
│   ├── menu.sass
│   ├── message.sass
│   ├── modal.sass
│   ├── navbar.sass
│   ├── pagination.sass
│   ├── panel.sass
│   └── tabs.sass
├── elements
│   ├── _all.sass
│   ├── box.sass
│   ├── button.sass
│   ├── container.sass
│   ├── content.sass
│   ├── form.sass
│   ├── icon.sass
│   ├── image.sass
│   ├── notification.sass
│   ├── other.sass
│   ├── progress.sass
│   ├── table.sass
│   ├── tag.sass
│   └── title.sass
├── grid
│   ├── _all.sass
│   ├── columns.sass
│   └── tiles.sass
├── layout
│   ├── _all.sass
│   ├── footer.sass
│   ├── hero.sass
│   └── section.sass
└── utilities
├── _all.sass
├── animations.sass
├── controls.sass
├── derived-variables.sass
├── functions.sass
├── initial-variables.sass
└── mixins.sass
I've tried to edit navbar.sass:
$navbar-item-hover-background-color: #28567d !default
and then rebuild the bulma.css file with sass bulma.sass bulma.css
It had no effect though.
Any assistance much appreciated.

tree terminal command: Avoid printing all sub folders/files and putting limit

I would like to print all the subdirectories and files from a certain directory. But some of the subfolders have humungous number of files and I would like to cap the number of subdirectories/files they print for each subfolder where it goes over that cap. How do I do it?
Currently I have this situation:
/data$ tree
.
├── filenames.json
├── tripletlists
│   ├── class_tripletlist_test.txt
│   ├── class_tripletlist_train.txt
│   ├── class_tripletlist_val.txt
│   ├── closure_tripletlist_test.txt
│   ├── closure_tripletlist_train.txt
│   ├── closure_tripletlist_val.txt
│   ├── gender_tripletlist_test.txt
│   ├── gender_tripletlist_train.txt
│   ├── gender_tripletlist_val.txt
│   ├── heel_tripletlist_test.txt
│   ├── heel_tripletlist_train.txt
│   └── heel_tripletlist_val.txt
└── ut-zap50k-images
├── Boots
│   ├── Ankle
│   │   ├── adidas
│   │   │   ├── 8030969.3.jpg
│   │   │   └── 8030970.107722.jpg
│   │   ├── adidas Kids
│   │   │   ├── 8070145.388249.jpg
│   │   │   └── 8070146.388250.jpg
│   │   ├── adidas Originals
│   │   │   ├── 8027274.372160.jpg
│   │   │   ├── 8027274.372161.jpg
│   │   │   ├── 8027310.115329.jpg
│   │   │   ├── 8027310.183092.jpg
│   │   │   ├── 8027320.372147.jpg
│   │   │   └── 8027320.372178.jpg
│   │   ├── adidas Originals Kids
│   │   │   ├── 8025627.371498.jpg
│   │   │   ├── 8025627.74095.jpg
│   │   │   ├── 8025719.11196.jpg
You can use the flag --filename N in tree --filenames N where N is the number of caps. For example, if I just want to print maximum of four subdirectories or files per subdirectory, you can youse tree --filename 4.
>> ls
filenames.json tripletlists ut-zap50k-images
>> tree --filelimit 4
.
├── filenames.json
├── tripletlists [12 entries exceeds filelimit, not opening dir]
└── ut-zap50k-images
├── Boots [5 entries exceeds filelimit, not opening dir]
├── Sandals
│   ├── Athletic [6 entries exceeds filelimit, not opening dir]
│   ├── Flat [314 entries exceeds filelimit, not opening dir]
│   └── Heel [25 entries exceeds filelimit, not opening dir]
├── Shoes [10 entries exceeds filelimit, not opening dir]
└── Slippers
├── Boot [6 entries exceeds filelimit, not opening dir]
├── Slipper Flats [77 entries exceeds filelimit, not opening dir]
└── Slipper Heels
├── Daniel Green [8 entries exceeds filelimit, not opening dir]
└── L.B. Evans
├── 7590239.255.jpg
└── 7590239.72.jpg

Generate a list or map of css files

I'm starting to work on a new app at my company. I'm hoping to run a quick process that will generate an outline, tree, or other map-type thing of all of the CSS and SASS files in the app directory.
I know I can grep it, but I wanted to see if someone had something more targeted I could use.
If you're simply looking to generate a tree, the common tree command can filter by file type if provided a pattern. Maybe this will help:
tree -P "*.css" --prune
The -P option allows you to match a pattern, and the --prune option hides empty folders (or ones which don't contain match files).
It's a pretty nifty tool; here's some sample output from tree -P "*.js" --prune on a node project directory:
.
├── Authorize.js
├── collections.js
├── functions
│   ├── downloadImage.js
│   ├── generateThumbnails.js
│   ├── hashImage.js
│   ├── loadMedia.js
│   └── uploadFile.js
├── node_modules
│   ├── body-parser
│   │   ├── index.js
│   │   ├── lib
│   │   │   ├── read.js
│   │   │   └── types
│   │   │   ├── json.js
│   │   │   ├── raw.js
│   │   │   ├── text.js
│   │   │   └── urlencoded.js
│   │   └── node_modules
│   │   ├── bytes
│   │   │   └── index.js
│   │   ├── content-type
│   │   │   └── index.js
More documentation here: http://www.computerhope.com/unix/tree.htm

Custom grunt configuration

I'm porting an application from php to node(sailsjs) at the same time trying to replace ant with grunt. I like the current project build structure and I would like to preserve some of it.
It looks like below...
project root
├── build (git ignored)
│   ├── coverage
│   ├── dist(to be deployed to target env)
│   └── local(to be deployed to local env)
├── lib
│   └── some library files like selenium..etc.
├── src
│   ├── conf
│   │   └── target/local properties
│   ├── scripts(may not be needed with grunt??)
│   │   ├── db
│   │   │   └── create_scripts...
│   │   ├── se
│   │   │   └── run_selenium_scripts...
│   │   └── tests
│   │   └── run_unit_test_scripts...
│   ├── tests
│   │   └── test_code....
│   └── webapp(this is where I'd like to place node[sailsjs] code)
│      └── code....
└── wiki .etc...
It doesn't exactly have to be the same way as above but more or less I prefer to build something similar. Now, pretty much all the sailsjs examples I have seen look like below.
project root
├── .tmp
│   └── stuff...
├── package.json
├── tasks
│   ├── config
│   │   └── grunt task configs...
│   └── register
│      └── grunt task registrations...
├── tests
│   ├── unit
│   └── selenium
└── Gruntfile.js
Where should I place Gruntfile.js, app.js, package.json to achieve what I want? What other detail should I have to make grunt function and create artifacts as I want them?
Note: Obviously I'm not expecting to get all the details of grunt configuration. But I guess it helps to see where most important things go and how basic tasks could be configured.
Thanks for your answer.
It's hard to give a precise answer without a detail of your build steps, but I would suggest:
Gruntfile.js and package.json go to your root folder
you setup your individual build tasks (whatever they are) to output to build: see the doc of each task on how to do that, it's usually the dest option
Hope this helps a bit.

Resources