I'm setting up an AWS MWAA instance and I have a problem with import custom plugins.
My local project structure looks like this:
airflow-project
├── dags
│ └── dag1.py
└── plugins
├── __init__.py
└── operators
├── __init__.py
└── customopertaor.py
I tried to match this structure in the s3 bucket:
s3://{my-bucket-name}
└── DAGS
├── dags
│ └── dag1.py
└── plugins
├── __init__.py
└── operators
├── __init__.py
└── customopertaor.py
However when I use the custom operator on the local project the import works like this -
from operators import customOperators
and on the MWAA it only recognize imports like this -
from plugins.operators import customOperators
Is there a way to get the MWAA recognize the import as the local (from operators)?
should I upload the files in certain way to the s3?
I also tried to upload a plugins.zip file but it didn't work:
s3://{my-bucket-name}
├── DAGS
│ └── dags
│ └── dag1.py
└── plugins.zip
I believe the proper way is to place your custom python modules in the plugin.zip file. This file will be uploaded to MWAA and gets extracted to /usr/local/airflow/plugins/. I believe the DAGs are placed in the very same folder.
AWS has published a User Guide that gives some good explanation and examples.
I had the same problem and i solve it looking inside my .zip file. In my case the structure inside .zip file creates an extra folder called plugins. Review this using unzip -l plugins.zip and look the tree generated. This is my working structure:
Archive: plugins.zip
Length Date Time Name
0 10-18-2021 11:39 hooks/
125 10-18-2021 11:40 hooks/my_airflow_hook.py
0 10-18-2021 11:40 sensors/
359 10-18-2021 11:40 sensors/my_airflow_sensor.py
395 10-18-2021 13:28 my_airflow_plugin.py
0 10-18-2021 11:42 operators/
437 10-18-2021 11:42 operators/hello_operator.py
480 10-18-2021 11:42 operators/my_airflow_operator.py
you can import the plugin as a python-module like below
import imp
customopertaor = imp.load_source('customopertaor','/usr/local/airflow/plugins/operators/customopertaor.py')
Your plugin folder tree looks good.
You need to restart your airflow environment to take the new plugin into account.
Alternatively you can use the config reload_on_plugin_change.
When I'm developing locally the images are found when I place the /public folder in /src/:
# locally
./src/public/images/
Then when I do a deploy the images are not found. But when I place the public folder outside of src the images are found:
# deploy
./public/images/
./src/
And I use the images as:
<img src="/images/my-image.jpg" alt="" />
Is there a configuration setting I have to use?
Edit:
Full structure:
|- .now/
| |- project.json
| └── README.txt
|- next.config.js
|- now.json
|- src/
| |- .next/
| | |- build-manifest.json
| | |- react-loadable-manifest.json
| | |- cache/
| | |- server/
| | └── static/
| |- pages/
| └── next-env.d.ts
The public folder has to be in the root. There’s no way to configure otherwise.
Files inside public can then be referenced by your code starting from the base URL (/).
/public/path/image.jpg is served as /path/image.jpg
https://nextjs.org/docs/basic-features/static-file-serving
Next.js can serve static files, like images, under a folder called public in the root directory.
You can check the document here
import Image from 'next/image'
function Avatar() {
return <Image src="/me.png" alt="me" width="64" height="64" />
}
export default Avatar
Have fun.
You might have problem with letter case like .png and .PNG. You have to use the same case in code as in the image extension case.
I'm having this rather annoying trouble that I'm having some issues understanding when my Sass compiles.
I have a SCSS folder that compiles to a CSS folder in the root of my site, but in my SCSS folder I have individual folders for layout, utilities, etc.
The issue I am having is within my SCSS structure I have a vendors folder that houses everything from Bootstrap Grid, Font Awesome, etc. To keep the structure of the vendors folder neat and tidy I like to keep each vendor add on in a seperate folder. I use the following watch command:
sass --watch scss:css
Here's the file structure where ... is the files within the folder.
project-name/
├── scss/
│ └── style.scss
└── vendors/
├── bootstrap-grid
│ └── ...
└── fontawesome
└── ...
The issue I have when compiling to the CSS folder, the vendors folder and its contents are being compiled to the CSS folder:
project-name/
├── css/
│ └── style.css
└── vendors/
├── bootstrap-grid
│ └── ...
└── fontawesome
└── ...
Thanks in advance :)
Try using this command:
sass --watch scss/style.scss:css/style.css
I really like eslint for es6 projects. Previously I've used it for new projects. Now I want to add it to a legacy project.
Fixing all pre-existing lint issues in one go is too much effort. Can I configure eslint (in .eslintrc.js) to only check files where I've explicitly enabled it with /* eslint-enable */ or similar?
ESLint has no default-disabled state that can be toggled by a file comment. You might be able to use .eslintignore for this purpose, however. You can ignore everything and then gradually whitelist files as you migrate them by using ! to un-ignore individual files. For example:
.
├── .eslintignore
├── .eslintrc.js
├── package.json
├── node_modules
│ └── ...
├── src
│ ├── index.js
│ └── module
│ └── foo.js
└── yarn.lock
Then your .eslintignore could look something like this:
# Start by ignoring everything by default
src/**/*.js
# Enable linting just for some files
!src/module/foo.js
In this case, src/index.js would be ignored, but it would lint src/module/foo.js.
I'm new to Django, and am trying to set up a really simple Django app.
Now, I'm up to chapter 5 in the Django online book : http://www.djangobook.com/en/2.0/chapter05/
All I want to do now, before I start trying databases, is to add in some simple CS and J to the app as is.
So the question is, how do I do this? I only have one app, and I only want a main.css in a css folder, and a main.js in a js folder.
I checked out the https://docs.djangoproject.com/en/1.3/howto/static-files/#staticfiles-in-templates page, but after reading and reading, there didn't seem to be a whole lot to work with in terms of examples.
How do I import the stylesheets/css (is there a helper like CakePHP?), where do I put the CSS and JS files, and do I need to configure the static thingy?
UPDATE: the link : Render CSS in Django does not help much. Mainly in that it didn't work for me, but also I especially don't like the "if debug" part. What happens when I move to prod? Why would I have my media folder defined differently for prod and local dev anyway? Shouldn't it be relative, or even in the same spot?
If you follow django's guidelines, you can simplify your life greatly.
In your sample code, inside your application directory, create a folder called static. Inside this folder, place your css files.
Example:
$ django-admin.py startproject myproject
$ cd myproject
myproject$ python manage.py startapp myapp
myproject$ mkdir myapp/static
myproject$ cd myapp/static
myproject/myapp/static$ nano style.css
In your templates:
<link rel="stylesheet" href="{{ STATIC_URL }}style.css" />
Make sure you add myapp to the INSTALLED_APPS list in settings.py. Now when you use the built-in development server, your style sheet will be rendered correctly.
Django searches for a static directory inside installed applications by default, and with current versions of django, static files are enabled by default.
The Django example has the path my_app/static/my_app/myimage.jpg which
is a little confusing if your app and project have the same name.
This is recommended because when you run collectstatic to gather all your static files, files with the same name will be overwritten. If you have a file called myimage.jpg in another application, it will be overwritten. Giving the application name inside the static directory will prevent this, because the exact directory structure will be replicated inside your STATIC_ROOT directory.
A simple example to illustrate the point. If you have a django project with two apps, like this:
.
├── assets
├── manage.py
├── myapp
│ ├── __init__.py
│ ├── models.py
│ ├── static
│ │ └── myapp
│ │ └── test.txt
│ ├── tests.py
│ └── views.py
├── myproj
│ ├── __init__.py
│ ├── __init__.pyc
│ ├── settings.py
│ ├── settings.pyc
│ ├── urls.py
│ └── wsgi.py
└── otherapp
├── __init__.py
├── models.py
├── static
│ └── otherapp
│ └── test.txt
├── tests.py
└── views.py
assets is your STATIC_ROOT. Now when you run collectstatic:
.
├── assets
│ ├── myapp
│ │ └── test.txt
│ └── otherapp
│ └── test.txt
├── manage.py
├── myapp
│ ├── __init__.py
│ ├── __init__.pyc
│ ├── models.py
│ ├── static
│ │ └── myapp
│ │ └── test.txt
│ ├── tests.py
│ └── views.py
├── myproj
│ ├── __init__.py
│ ├── __init__.pyc
│ ├── settings.py
│ ├── settings.pyc
│ ├── urls.py
│ └── wsgi.py
└── otherapp
├── __init__.py
├── __init__.pyc
├── models.py
├── static
│ └── otherapp
│ └── test.txt
├── tests.py
└── views.py
You see it is creating the directories as well. In your templates you would now refer to each file with its "namespace" of the app: {{ STATIC_URL }}/myapp/test.txt
The recommended approach has changed again (from at least Django 1.5 I think).
At the top of your template put:
{% load staticfiles %}
Then, using the same directory structure (myapp/static/myapp/style.css):
<link rel="stylesheet" href="{% static 'myapp/style.css' %}" />
Source
For the examples you pointed at to work, your static files need to be in a location accessible to Django's built-in staticfiles app.
There are a couple steps to make this happen:
First, within your project directory (ie beside your manage.py file), you'll need to create a directory to hold your static files. Call it "static_files".
Next, you'll need to let Django know to look in that directory, by specifying it in the list of STATICFILES_DIRS within your settings.py file.
Something like this:
STATICFILES_DIRS = [
'/full/path/to/your/project/static_files/',
]
Within that static_files directory, you can create whatever structure you want, so that is where your css and js directories could go.
After that, you should be able to use the {{ STATIC_URL }} tag in your templates to get access to the base URL of your static files.
So, say, for example, you create project/static_files/css/base.css, you would use it in your template like so:
<link rel="stylesheet" type="text/css" href="{{ STATIC_URL }}/css/base.css" />
Hope that helps!
Edit
With the default settings for STATICFILES_FINDERS, Django should automatically serve up any files from directories listed in your STATICFILES_DIRS -- see the docs for details.
If this doesn't work, some things to check:
Have you edited your STATICFILES_FINDERS setting to something other than the default?
Is django.contrib.staticfiles in your list of INSTALLED_APPS in settings.py?
Are you using Django's built-in server (python manage.py runserver)?
Do you have DEBUG = True in your settings.py? If not, you'll need to either set it to True or use the insecure option (python manage.py runserver --insecure). When going to production, check out the collectstatic command.
finally i got after many days here is how should be done
1> you need your static file in your app along with in your project
2> here is setting.py
if DEBUG:
STATIC_ROOT = "/PycharmProjects/don/admin/shopping/static/css"
STATICFILES_DIRS =(
#os.path.join(os.path.dirname(BASE_DIR),"static","static"),
os.path.join(BASE_DIR, "static"),
)
3>views.py
from django.views.generic import TemplateView
class HomeView(TemplateView):
template_name = 'index.html'
4> template file
{% load staticfiles %}
<link rel="stylesheet" href="{% static 'css/style.css' %}">
5> urls.py
from shopping.views import HomeView
from django.conf import settings
from django.conf.urls.static import static
urlpatterns = [
url(r'^admin/', admin.site.urls),
url(r'^$', HomeView.as_view())
]
if settings.DEBUG:
urlpatterns+=static(settings.STATIC_URL,document_root=settings.STATIC_ROOT)
6> python manage.py collectstatic
have fun :)
The handling of static files has changed significantly (for the better) in Django 1.3. In particular is the flexible difference between static files (think code required CSS/JS/images) and media files (think images uploaded by users in the use of you application). The staticfiles app handles what you are asking for.
Put django.contrib.staticfiles in your settings.py INSTALLED_APPS. Then create a static directory inside your app directory - so project/app/static/. Within that static directory organize your support files as you like (css/, js/, images/, icons/, etc). staticfiles by default will look in the static/ directory for every app listed in INSTALLED_APPS. Not just your own, but all the Django apps and third-party apps. (sidenote: the Admin in Django 1.4 moves to this paradigm while in 1.3 it still uses the ADMIN_MEDIA_PREFIX).
The staticfiles app knows about all those static/ directories in all the apps but it needs to collect all that content in order to serve it. In development, when using manage.py runserver it is handled for you. Django will run your site and automatically deliver all the static content from all the static sources. (Sources, as mentioned in another answer, are set in the STATICFILES_FINDERS setting.) When not using runserver, use manage.py collectstatic to gather all the static files into the folder defined in STATIC_ROOT. Keep this directory empty. It is a collection destination. Of course your web server will need to be configured to serve this directory.
Now there is one more piece - the view. This is where the STATIC_URL comes into play. When referring to static files in your templates, the easiest way is to use {{ STATIC_URL }}. Django by default makes that variable available to any view using the RequestContext. Do something like this - <link rel="stylesheet" href="{{ STATIC_URL }}/css/main.css" />.
For more information and more ways to refer to STATIC_URL in your templates, check out this answer to a similar question.
Here's a hacky workaround for views.py when debug in settings.py is False
def my_style(request):
my_file = open('path to arbitrary css file', 'r')
response = HttpResponse(content=my_file.read())
response['Content-Type'] = 'text/css'
return response
and here's the urls.py
urlpatterns = [
path('hello', views.hello_world, name='hello_world'),
path('static/my_style/', views.my_style, name='my_style'),
]
And here's the insert for the html file
<link rel="stylesheet" type="text/css" href="static/my_style">
Though this is clearly not how Django was meant to be used, and I'm guessing this could have some security implications.