I am having trouble getting the contents of my INST folder to copy over to the root folder of the package (or copy over in general)
My R package depends on two java packages and their jars, these are both included as follows:
PackageTest
+-- inst
| +-- impala-jdbc-cdh5
| +-- *.jar
| +-- sqlserver-jdbc-4
| +-- *.jar
+-- R
| +-- hello.R
+-- man
| +-- hello.Rd
+-- DESCRIPTION
+-- NAMESPACE
+-- .Rbuildignore
From my understanding the contents of the Package/inst folder should be moved to the root of the folder on install resulting in a folder structure that looks like this:
PackageTest
+-- impala-jdbc-cdh5
| +-- *.jar
+-- sqlserver-jdbc-4
+-- *.jar
+-- R
| +-- PackageTest
| +-- PackageTest.rdb
| +-- PackageTest.rdx
+-- Meta
+-- html
+-- help
+-- DESCRIPTION
+-- NAMESPACE
+-- INDEX
Instead I am missing the two folders listed at the top:
PackageTest
+-- R
| +-- PackageTest
| +-- PackageTest.rdb
| +-- PackageTest.rdx
+-- Meta
+-- html
+-- help
+-- DESCRIPTION
+-- NAMESPACE
+-- INDEX
There are other files in the other folders, but nothing that I am concerned with.
What I have tried:
I am using RStudio to generate this package and all I am doing is modifying the DESCRIPTION package to include the two Java packages. (so it really is a stub package with some Java jars - I have also imported rJava)
To build and install the package I have tried using devtools::build and devtools:install and the shortcuts that RStudio provides (Ctrl + Shift + B).
I have also tried to upload this repo onto our internal Git repo and use devtools::install_git and that results in the same issue.
Restructuring the inst folder to so that all the jars are in inst/Java has not helped with my issue.
Lastly, the package we have deployed on our production repo is bundled the exact same as the first example and we have no issues using the impala and sql drivers.
Any help would be appreciated. :)
EDIT:
.Rbuildignore
^.*\.Rproj$
^\.Rproj\.user$
Related
Themosis Framework provides (laravel like) MVC type development environment for wordpress.
It all look great, but just got stuck in developing a "stand alone" plugin, because plugin look for configuration files in higher level directory.
This is how root directory look like
+-- app/
| +-- Console
| +-- Exceptions
| +-- Forms
| +-- Hooks
| +-- Http
| +-- Controllers
| +-- Mail
| +-- Providers
| +-- Widgets
+-- bootstrap/
+-- config/
| +-- app.php
| +-- ...
| +-- wordpress.php
+-- database/
+-- htdocs/
| +-- cms/
| +-- content/
| +-- index.php
| +-- wp-config.php
+-- resources/
| +-- languages/
| +-- views/
+-- routes/
| +-- console.php
| +-- web.php
+-- storage/
+-- tests/
+-- vendor/
+-- .env
+-- composer.json
+-- console
+-- wp-cli.yml
this is how plugin directory look like in /htdocs/content/plugins:
+-- assets
+-- config
+-- dist
+-- inc
+-- languages
+-- resources
+-- views
+-- package.json
+-- plugin-name.php
+-- routes.php
+-- webpack.mix.js
Themosis has interesting feature of compiling (although its php based framework) so I thought compiling will create some asset to make plugin work as standalone plugin.
but it seems error (plugin looking for resources stored in higher level directory) still remain
Would anyone know if making "standalone" plugin is possible using themosis framework?
Thank you!
As my project grows I'm wanting to setup an elegant and scalable folder structure but I'm unsure what the best practices are.
My project includes a
Flutter app
Firebase Functions
Firebase Hosting with a static landing page and will soon have 2 Angular apps
My current structure is this:
Root flutter app
|
+-- README.md
+-- package.json (with flutter build commands)
+-- Flutter code (lib, test assets etc)
|
+-- Firebase
| |
| +-- package.json (with firebase build commands)
| +-- Functions
| | |
| | +-- src (functions code)
| | |
| +-- Firestore
| | |
| | +-- firestore.rules
| | +-- firestore.indexes
Should my root directory look like this:
- Flutter
- Firebase (Functions, hosting, firestore)
Or something more like this:
- Flutter
- Functions
- Hosting (App1, App2, Static landing etc)
- Firestore
But then where is best to keep build files and deploy scripts for each site/service/app?
I haven't implemented CI/CD yet, but I plan to implement it soon, so I'd probably want to take that into consideration.
Thanks for any advice!
This is actually two questions combined into one.
My AIRFLOW_HOME is structured like
airflow
+-- dags
+-- plugins
+-- __init__.py
+-- hooks
+-- __init__.py
+-- my_hook.py
+-- another_hook.py
+-- operators
+-- __init__.py
+-- my_operator.py
+-- another_operator.py
+-- sensors
+-- utils
I've been following astronomer.io's examples here https://github.com/airflow-plugins. My custom operators use my custom hooks, and all the imports are relative to the top level folder plugins.
# my_operator.py
from plugins.hooks.my_hook import MyHook
However, when I tried moving my entire repository into the plugins folder, I get an import error after running airflow list_dags saying that plugins cannot be found.
I read up a little about it and apparently Airflow loads the plugins into its core module so they can be imported like
# my_operator.py
from airflow.hooks.my_hook import MyHook
So I changed all the imports to read directly from airflow.plugin_type instead. I get another import error though, this time saying that my_hook cannot be found. I restart my workers, scheduler and webserver every time but that doesn't seem to be the issue. I've looked at solutions proposed in similar questions and they don't work either.
The official documentation also shows this way https://airflow.apache.org/plugins.html of extending the AirflowPlugin class, but I'm not sure where this "interface" should reside. I would also prefer a drag and drop option.
Finally, it clearly doesn't make sense for my code repo to be the plugins folder itself, but if I separate them testing becomes inconvenient. Do I have to modify my Airflow configurations to point to my repo every time I run unit tests on my hooks/ops? What are the best practices for testing custom plugins?
I figured this out by doing some trial and error. This is the final structure of my AIRFLOW_HOME folder
airflow
+-- dags
+-- plugins
+-- __init__.py
+-- plugin_name.py
+-- hooks
+-- __init__.py
+-- my_hook.py
+-- another_hook.py
+-- operators
+-- __init__.py
+-- my_operator.py
+-- another_operator.py
+-- sensors
+-- utils
In plugin_name.py, I extend the AirflowPlugin class
# plugin_name.py
from airflow.plugins_manager import AirflowPlugin
from hooks.my_hook import *
from operators.my_operator import *
from utils.my_utils import *
# etc
class PluginName(AirflowPlugin):
name = 'plugin_name'
hooks = [MyHook]
operators = [MyOperator]
macros = [my_util_func]
In my custom operators which use my custom hooks, I import them like
# my_operator.py
from hooks.my_hook import MyHook
Then in my DAG files, I can do
# sample_dag.py
from airflow.operators.plugin_name import MyOperator
It is necessary to restart the webserver and scheduler. Took me a while to figure out.
This also facilitates testing since the imports within the custom classes are relative to the sub modules within the folder plugins. I wonder if I can omit the __init__.py file inside plugins, but since everything is working I didn't try doing that.
I have the following R packages:
ubuntu#ip-111-31-16-140:~/storage1/homebrew_rpackages/mypackr$ pwd
/home/ubuntu/storage1/homebrew_rpackages/mypackr
The structure is:
ubuntu#ip-172-31-16-140:~/storage1/homebrew_rpackages/mypackr$ tree
.
|-- DESCRIPTION
|-- NAMESPACE
|-- R
| |-- hello.R
| `-- tsne_pca.R
|-- mypackr.Rproj
`-- man
|-- hello.Rd
`-- plot_tsne_pca.Rd
What I tried to do is to install locally this way, but gives error:
> devtools::install_local("/home/ubuntu/storage1/homebrew_rpackages/mypackr")
Installation failed: trying to get slot "name" from an object of a basic class ("NULL") with no slots
What's the right way to do it?
You have to do at least one commit in your git repository of your package.
I can use Maven with FlexMojos to generate swf file, but because I have several mxml files need to build, could I build them in 1 Maven pom file?
I know if I separate them into several projects, Maven can build them. But because these mxml files have some shared functions, it's not easy to manage if I separate them.
Another question is the size of swf file. The original file size build by Flex builder is about 80KB, but the file size by Maven is about 800KB. Is it normal?
This sounds to me like a multi-module build which is exactly intended for such purposes.
+-- pom.xml
+-- module1
+--- pom.xml
+-- module2
+--- pom.xml
+-- common
+--- pom.xml
I can recommend the Sonatype book about this.
You can create a sub-module which contains several flex modules...
+-- pom.xml
+-- module1
+--- pom.xml
+-- module2
+--- pom.xml
+-- module-flex
+-- pom.xml
+-- mod-flex-1
+-- pom.xml
+-- mod-flex-2
+-- pom.xml
+-- mod-flex-common
+-- pom.xml
+-- common
+--- pom.xml