Rails 6 Mina deploy fails at Precompile - ruby-on-rails-6

I am running a Debian 10 (Buster) Server for my Rails 6.0.3, Ruby 2.6.3, Mina 1.2.3 app. When I run Mina deploy it errors out but does not give me enough to help me understand what I need to do to resolve the issue.
Mina Deploy output
Precompiling asset files
$ RAILS_ENV="production" bundle exec rake assets:precompile
Traceback (most recent call last):
3: from /var/www/site/tmp/build-15965768145748/vendor/bundle/ruby/2.6.0/bin/ruby_executable_hooks:24:in `<main>'
2: from /var/www/site/tmp/build-15965768145748/vendor/bundle/ruby/2.6.0/bin/ruby_executable_hooks:24:in `eval'
1: from /usr/local/rvm/gems/ruby-2.6.3/bin/rake:23:in `<main>'
/usr/local/rvm/gems/ruby-2.6.3/bin/rake:23:in `load': cannot load such file -- /usr/local/rvm/rubies/ruby-2.6.3/lib/ruby/gems/2.6.0/specifications/default/exe/rake (LoadError)
! ERROR: Deploy failed.
Deploy.rb
require 'mina/bundler'
require 'mina/rails'
require 'mina/git'
require 'mina/rvm'
set :application_name, 'site'
set :domain, 'server'
set :user, 'deployer'
set :deploy_to, "/var/www/site"
set :repository, "git#github.com:git.git"
set :branch, 'master'
set :rvm_use_path, '/etc/profile.d/rvm.sh'
set :shared_files, fetch(:shared_files, []).push('config/database.yml', 'config/secrets.yml')
task :remote_environment do
ruby_version = File.read('.ruby-version').strip
raise "Couldn't determine Ruby version: Do you have a file .ruby-version in your project root?" if ruby_version.empty?
invoke :'rvm:use', 'ruby-2.6.3'
end
desc "Deploys the current version to the server."
task :deploy do
deploy do
invoke :'git:clone'
invoke :'deploy:link_shared_paths'
invoke :'bundle:install'
invoke :'rails:db_migrate'
invoke :'rails:assets_precompile'
invoke :'deploy:cleanup'
on :launch do
end
end
end
namespace :passenger do
desc "Restart Passenger"
task :restart do
queue %{
echo "-----> Restarting passenger"
cd #{deploy_to}/current
#{echo_cmd %[mkdir -p tmp]}
#{echo_cmd %[touch tmp/restart.txt]}
}
end
end
I have made sure that yarn is installed and working on my Production machine. When I run the precompile on the Production machine, it runs through successfully. But still errors out in the Mina Deploy with no change in the error text. I have looked into this article In doing so I have commented out the suggested code snippet just to test, but that did not change anything. In looking at my bashrc, I do not see any snippets with EXPORT or SOURCE at the beginning of them.
EDIT: When I run the precompile on the server side, each new deployment folder seems like it will give me this issue.

Related

Vague error in Vercel deployment (exit with code 1)

So normally Vercel gives me decent error messages, but this one seems to have no detail. Just
"failed with exit code 1".
For context, I just started this project w the basic NextJS template, made a few modifications, and this is my first deploy.
Failed once and I assumed it was because NextJS has strict type checking on by default, so I modified the next.config.js file as follows (per here):
module.exports = {
typescript: {
// !! WARN !!
// Dangerously allow production builds to successfully complete even if
// your project has type errors.
// !! WARN !!
ignoreBuildErrors: true,
},
};
But still failed. Here is the full log:
10:23:27.575 Cloning github.com/lawderp/physical (Branch: main, Commit: 8f97554)
10:23:28.589 Cloning completed: 1.014s
10:23:28.614 Analyzing source code...
10:23:29.739 Installing build runtime...
10:23:31.612 Build runtime installed: 1.873s
10:23:34.043 Looking up build cache...
10:23:34.164 Build cache found. Downloading...
10:23:35.100 Build cache downloaded [21.48 MB]: 936.007ms
10:23:36.250 Installing dependencies...
10:23:36.489 yarn install v1.22.11
10:23:36.547 [1/4] Resolving packages...
10:23:36.734 success Already up-to-date.
10:23:36.743 Done in 0.26s.
10:23:36.754 Detected Next.js version: 11.1.2
10:23:36.755 Running "yarn run build"
10:23:37.021 yarn run v1.22.11
10:23:37.047 $ next build
10:23:37.820 info - Using webpack 5. Reason: Enabled by default https://nextjs.org/docs/messages/webpack5
10:23:37.976 info - Skipping validation of types...
10:23:40.439 error Command failed with exit code 1.
10:23:40.439 info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
10:23:40.456 Error: Command "yarn run build" exited with 1
Still not sure what the issue was but eventually figured out how to get past this - ignored ESLint during build by adding the following to next.config.js:
eslint: {
ignoreDuringBuilds: true,
},
(I don't advise doing this but this was quick low-stakes experiment)
I received the same vague message after making eslint alterations. Locally linting and deployment was successful, however deploying to vercel failed like yours.
Although "ignoreDuringBuilds": "true" in my .eslint.json did band-aid fix the issue, I was able to rollback each eslint change and found that my "linebreak-style": ["error", "windows"] was the underlying issue.
It passed linting locally, but failed on vercel deployment. I've fixed my line-breaks and all works great now without having to ignore during builds.

Google Cloud Composer DataflowJavaOperator: 403 Forbidden When Creating Job in Another Project

I am trying to use DataflowJavaOperator on our testing composer environment, but I am running into a 403 forbidden error. My intention is to kick off a Dataflow Java job on a different project using the test composer environment.
t2 = DataFlowJavaOperator(
task_id = "run-java-dataflow-job",
jar="gs://path/to/dataflow-jar.jar",
dataflow_default_options=config_params["dataflow_default_options"],
gcp_conn_id=config_params["gcloud_config"]["conn_id"],
dag=dag
)
My default options look like
'dataflow_default_options': {
'project': 'other-project',
'input': 'other-project:dataset.table',
'output': 'other-project:dataset.table'
...
}
I have tried creating a temporary composer test environment in the same project as the Dataflow, and this allows me to use DataflowJavaOperator as expected. Only when the composer environment resides in a different project as the Dataflow, does DataflowJavaOperator not work as expected.
My current workaround is to use BashOperator, use "env" to set the GOOGLE_APPLICATION_CREDENTIALS as gcp_conn_id path, store the jar file in our test composer bucket, and just run this bash command:
java -jar /path/to/dataflow-jar.jar \
[... all Dataflow job options]
Is it possible to use DataflowJavaOperator to kick off Dataflow jobs on another project?
You need a different GCP connection created for Composer to interact with your 2nd GCP project and you need to pass that connection id to gcp_conn_id in DataFlowJavaOperator

DNX Webjob fails to resolve local project dependencies

We have a Solution created with VS2015.1 that has 3 projects, all of them DNX RC1.
| sln
|+---MVC6 Api
| project.json
| +---ClassLibrary
| Class.cs
| project.json
| \---Webjob
Program.cs
project.json
Both the API and Webjob reference the ClassLibrary in their project.json files.
"frameworks": {
"dnx451": {
"dependencies": {
"ClassLibrary": "1.0.0-*"
}
}
}
We created 2 WebApps, both tied with Continuos Integration to the Solution repo and each with a Project Application Setting that points to the API folder (in one) and the Webjob folder (in the other).
The API project gets deployed without problem, when it tries to resolve the local project dependency it works:
Using Project dependency ClassLibrary 1.0.0 for DNX,Version=v4.5.1
Copying source code from Project dependency ClassLibrary
Source D:\home\site\repository\ClassLibrary\project.json
Target D:\local\Temp\8d32044390806ef\approot\src\ClassLibrary
The Webjob on the other hand, fails, it tries to resolve it as a nuget remote package and fails:
[01/21/2016 19:47:45 > b597c3: INFO] GET https://api.nuget.org/v3-flatcontainer/ClassLibrary/index.json
[01/21/2016 19:47:46 > b597c3: INFO] NotFound https://api.nuget.org/v3-flatcontainer/ClassLibrary/index.json 507ms
[01/21/2016 19:47:46 > b597c3: ERR ] Unable to locate Dependency ClassLibrary >= 1.0.0-*
This last part I had to check it through Kudu, since it's done on the first run of the WebJob apparently and not during the deploy phase (like with the API).
Running the WebJob locally works without issues.
I tried the solution in publish dnx based WebJob with local dependencies but didn't work.
After talking with the Product Team, the 3 project scenario won't work right now by default.
The solution described on publish dnx based WebJob with local dependencies works for a 2 project scenario, but if you need to deploy a WebJob AND a Web App that depend on the same Class Library project and the 3 are on the same repo, you can't make the Webjob deploy from Continuous Integration.
The solution is to set Continuous Integration for your Web App (it will work by default) and manually deploy your WebJob as a zipped file.
Go into your WebJob folder and run dnu publish.
Go into bin/output/approot/src/YourWebJobFolder.
Edit the autogenerated CMD file.
Use this script by customizing line 4:
#ECHO OFF
:: 1. Prepare environment
SET DNX_CONSOLE_APP_PATH=Autocosmos.Trunk.Webjob
SET DNVM_CMD_PATH_FILE="%USERPROFILE%\.dnx\temp-set-envvars.cmd"
:: 2. Install DNX
CALL PowerShell -NoProfile -NoLogo -ExecutionPolicy unrestricted -Command "[System.Threading.Thread]::CurrentThread.CurrentCulture = ''; [System.Threading.Thread]::CurrentThread.CurrentUICulture = '';$CmdPathFile='%DNVM_CMD_PATH_FILE%';& '%SCM_DNVM_PS_PATH%' " install latest
IF ERRORLEVEL 1 GOTO ERROR
:: 3. Put DNX on the path
IF EXIST %DNVM_CMD_PATH_FILE% (
CALL %DNVM_CMD_PATH_FILE%
DEL %DNVM_CMD_PATH_FILE%
)
call :ExecuteCmd dnx --project "%~dp0src\%DNX_CONSOLE_APP_PATH%" --configuration Release %DNX_CONSOLE_APP_PATH% run
goto end
:ExecuteCmd
setlocal
set _CMD_=%*
call %_CMD_%
if "%ERRORLEVEL%" NEQ "0" echo Failed exitCode=%ERRORLEVEL%, command=%_CMD_%
exit /b %ERRORLEVEL%
:error
endlocal
echo An error has occurred during web site deployment.
call :exitSetErrorLevel
call :exitFromFunction 2>nul
:exitSetErrorLevel
exit /b 1
:exitFromFunction
()
:end
endlocal
echo Finished successfully.
Then ZIP bin/output/approot and Upload it to Azure.

Command 'generate' not found, compiling with rebar

I am following this blog:
http://maplekeycompany.blogspot.se/2012/03/very-basic-cowboy-setup.html
In short, I am trying to compile an application with rebar just as the person in the blog.
Everything goes smoothly until I want to run the command:
./rebar get-deps compile generate
This then give me the following errors and warnings,
> User#user-:~/simple_server/rebar$ ./rebar get-deps compile generate
> ==> rebar (get-deps)
> ==> rebar (compile) Compiled src/simple_server.erl Compiled src/simple_server_http.erl src/simple_server_http_static.erl:5:
> Warning: behaviour cowboy_http_handler undefined Compiled
> src/simple_server_http_static.erl
> src/simple_server_http_catchall.erl:2: Warning: behaviour
> cowboy_http_handler undefined Compiled
> src/simple_server_http_catchall.erl WARN: 'generate' command does not
> apply to directory /home/harri/simple_server/rebar Command 'generate'
> not understood or not applicable
I have found a similar post with the same error:
Command 'generate' not understood or not applicable
I think the problem is in the reltool.config but do not know how to proceed, I changed the path to the following: {lib_dirs, ["home/user/simple_server/rebar"]}
Is there a problem with the path? How can rebar get access to all the src files and also the necessary rebar file to compile and build the application?
You need to make sure your directory structure and its contents are arranged so that rebar knows how to build everything in your system and generate a release for it. Your directory structure should look like this:
project
|
-- rel
|
-- deps
|
-- apps
|
-- myapp
| |
| -- src
| -- priv
|
-- another_app
The rel directory holds all the information needed to generate a release, and the apps directory is where the applications that make up your project live. Application dependencies live in the deps directory. Each app such as myapp and another_app under the apps directory can have their own rebar.config files. While two or more such applications are possible here, normally you'd have just one and all others would be dependencies.
In the top-level project directory there's also a rebar.config file with contents that look like this:
{sub_dirs, ["rel", "apps/myapp", "apps/another_app"]}.
{lib_dirs, ["apps"]}.
If necessary, you can use rebar to generate your apps from application skeletons:
cd apps
mkdir myapp another_app
( cd myapp && rebar create-app appid=myapp )
( cd another_app && rebar create-app appid=another_app )
If an application has dependencies, you'll have to add a rebar.config to its directory and declare each dependency there. For example, if myapp depends on application foo version 1.2, create apps/myapp/rebar.config with these contents:
{deps,
[{foo, "1.*", {git, "git://github.com/user/foo.git", {tag, "foo-1.2"}}}]
}.
When you run rebar get-deps, rebar will populate the top-level deps directory to hold all dependencies, creating deps if necessary. The top-level rebar.config can also declare dependencies if necessary.
You also need to generate a node, necessary for your releases:
cd ../rel
rebar create-node nodeid=project
You then need to modify the reltool.config file generated by the previous step. You need to change
{lib_dirs, []},
to
{lib_dirs, ["../apps", "../deps"]},
and just after the line {incl_cond, derived}, add {mod_cond, derived}, so that releases contain only the applications needed for correct execution.
Next, wherever the atom 'project' appears, you need to replace it with the applications under the apps directory. For our example, we'd change this part:
{rel, "project", "1",
[
kernel,
stdlib,
sasl,
project
]},
to this:
{rel, "project", "1",
[
kernel,
stdlib,
sasl,
myapp,
another_app
]},
and change this part:
{app, project, [{mod_cond, app}, {incl_cond, include}]}
to this:
{app, myapp, [{mod_cond, app}, {incl_cond, include}]},
{app, another_app, [{mod_cond, app}, {incl_cond, include}]}
You might also need to add the line:
{app, hipe, [{incl_cond, exclude}]},
to exclude the hipe application since sometimes it causes errors during release generation or when trying to run the release. Try without it first, but add it if you see errors related to hipe when generating a release, or if attempts to run the generated release result in this sort of error:
{"init terminating in do_boot",{'cannot load',elf_format,get_files}}
you'll need to add it.
With all this in place you can now execute:
rebar get-deps compile generate
and you should be able to successfully generate the release. Note that running rebar generate at the top level rather than in the rel dir will result in a harmless warning like this, which you can ignore:
WARN: 'generate' command does not apply to directory /path/to/project
Finally, you can run the release. Here's how to run it with an interactive console:
$ ./rel/project/bin/project console
Exec: /path/to/project/rel/project/erts-6.2/bin/erlexec -boot /path/to/project/rel/project/releases/1/project -mode embedded -config /path/to/project/rel/project/releases/1/sys.config -args_file /path/to/project/rel/project/releases/1/vm.args -- console
Root: /path/to/project/rel/project
Erlang/OTP 17 [erts-6.2] [source] [64-bit] [smp:8:8] [async-threads:10] [kernel-poll:false]
Eshell V6.2 (abort with ^G)
(project#127.0.0.1)1>
or you could run ./rel/project/bin/project start to start it in the background. Run ./rel/project/bin/project with no arguments to see all available options.

Unable to create OpenShift application using --from-code option

I am trying to create an OpenShift application using the --from-code option to grab the application code from GitHub. I've created two different OpenShift QuickStarts -- with one, the --from-code option works, and with the other, it doesn't work.
So clearly I'm doing something wrong in the QuickStart that isn't working. But I can't see what I'm doing wrong. I either get error 504 or an error occurred, neither of which tells me what the problem is, and there doesn't seem to be a verbose flag to get more details on the error.
Tests-Mac:~ testuser$ rhc app create sonr diy-0.1 http://cartreflect-claytondev.rhcloud.com/reflect?github=smarterclayton/openshift-redis-cart --from-code https://github.com/citrusbyte/SONR.git
The cartridge 'http://cartreflect-claytondev.rhcloud.com/reflect?github=smarterclayton/openshift-redis-cart' will be downloaded and installed
Application Options
-------------------
Domain: schof
Cartridges: diy-0.1, http://cartreflect-claytondev.rhcloud.com/reflect?github=smarterclayton/openshift-redis-cart
Source Code: https://github.com/citrusbyte/SONR.git
Gear Size: default
Scaling: no
Creating application 'sonr' ... Server returned an unexpected error code: 504
Tests-Mac:~ testuser$ rhc app create sonr diy-0.1 http://cartreflect-claytondev.rhcloud.com/reflect?github=smarterclayton/openshift-redis-cart --from-code https://github.com/citrusbyte/SONR.git
The cartridge 'http://cartreflect-claytondev.rhcloud.com/reflect?github=smarterclayton/openshift-redis-cart' will be downloaded and installed
Application Options
-------------------
Domain: schof
Cartridges: diy-0.1, http://cartreflect-claytondev.rhcloud.com/reflect?github=smarterclayton/openshift-redis-cart
Source Code: https://github.com/citrusbyte/SONR.git
Gear Size: default
Scaling: no
Creating application 'sonr' ...
An error occurred while communicating with the server. This problem may only be temporary. Check that you have correctly specified your
OpenShift server 'https://openshift.redhat.com/broker/rest/domain/schof/applications'.
Tests-Mac:~ testuser$
That's creating an application with --from-code using this repo: https://github.com/citrusbyte/SONR . If I use this repo it works flawlessly: https://github.com/citrusbyte/openshift-sinatra-redis
The code itself seems to be good, as I can create an empty new application, merge the SONR code in, and it works flawlessly.
What am I doing wrong?
UPDATE: I've worked around this issue by creating the app in two stages instead of doing it in one stage:
rhc app create APPNAME diy-0.1 http://cartreflect-claytondev.rhcloud.com/reflect?github=smarterclayton/openshift-redis-cart
cd APPNAME
git remote add github -f https://github.com/citrusbyte/SONR.git
git merge github/master -s recursive -X theirs
git push origin master
I'd still love to know why doing it in one step was failing, though.
#developercorey had the right idea.
I tried with a ridiculous timeout of 99999, and then got a different timeout error that I don't think I can change:
$ rhc app create APPNAME diy-0.1 http://cartreflect-claytondev.rhcloud.com/reflect?github=smarterclayton/openshift-redis-cart --from-code https://github.com/citrusbyte/SONR.git --timeout 99999
...
Creating application 'APPNAME' ...
The initial build for the application failed: Shell command '/sbin/runuser -s /bin/sh 5328a9385973ca70150002af -c "exec /usr/bin/runcon 'unconfined_u:system_r:openshift_t:s0:c5,c974' /bin/sh -c \"gear postreceive --init >> /tmp/initial-build.log 2>&1\""' exceeded timeout of 229
The fix I mentioned in my earlier update is working perfectly, and that's what I recommend anyone with a similar problem try -- I'm creating the app as empty without the --from-code option, and then merging in the code I wanted to use in a separate step:
rhc app create APPNAME diy-0.1 http://cartreflect-claytondev.rhcloud.com/reflect?github=smarterclayton/openshift-redis-cart
cd APPNAME
git remote add github -f https://github.com/citrusbyte/SONR.git
git merge github/master -s recursive -X theirs
git push origin master
It could be that the application takes to long to clone/setup, and the creation is timing out. Something you can try is to create the application without the --from-code, then clone it locally, and merge in your code from github, then do a git push. This operation has a much longer timeout period, and will also let you see what, if any, errors that you get since the application won't disappear if it doesn't succeed, unlike an app create.

Resources