I have an error when i start artifactory:
[art-exec-3] [ERROR] (o.j.s.c.EncryptionWrapperBase:185) - no matched algorithm and key for AES128 16uJT
[art-exec-3] [ERROR] (o.a.r.s.RepositoryServiceImpl:2626) - Failed to initialize remote repository '(name of maven repo)'. Repository will be blacked-out!
The context is:
I had a functional standalone artifactory.
I have deployed a HA artifactory on this
So with the same database and filestore
I have destroyed my standalone instance and create 2 instances for high availability.
And then i have the error above.
The artifactory service is up
i can request API but only GET, no possible changes like POST.
But no web interface reachable.
If i test a POST request:
{
"errors" : [ {
"status" : 500,
"message" : "Could not decrypt with artifactory key, due to: org.jfrog.security.crypto.KeyIdAlgCipherNotFound: no matched algorithm and key forAES128 16uJT"
} ]
It is obviously an AES128 artifactory_key problem.
But i know my artifactory_key (and my master_key) are good.
It's still the same like before HA, no reason it's different but maybe ... ?
so, what's wrong ?
Can artifactory_key change over time without my consent ?
How can i recover ?
In my database, i have a table named "configs" and artifactory_key is used to encrypt the config i think.
If i recover this table, maybe i can override ?
Except it's all encrypted, with the artifactory_key ? or the master_key ?
It's not usable as is.
Thanks to you
Is the artifactory.key in place and located at $ARTIFACTORY_HOME/etc/security/ of both nodes?
Based on the error message it seems as it fails to decrypt a string that contains "16uJT".
Can you search for this string in $ARTIFACTORY_HOME/etc/artifactory.config.latest.xml ?
In case it is, you may check what is the expected decrypted value of that entry and manually replace it. If you are not sure, you may try to leave this one blank for now and try to see if this helped.
In order to import the new config after the change, you will need to save the config as: "artifactory.config.import.xml" and then restart the server.
In case this doesn't help, kindly attach the full errors with possible stacktraces.
Thanks for your reply
The problem is: i have destroyed the old instance and recreate two new instances with HA licenses.
So i have an artifactory_key in $ARTIFACTORY_HOME/etc/security/ and it's the same on both instances. But apparently not the good one.
And the file $ARTIFACTORY_HOME/etc/artifactory.config.latest.xml is not the good one too, like instances are new.
It's a config by default, a void config.
There is no "16uJT" string.
That's why i ask if i can retrieve the good configuration directly in the database -> table "configs"
except it's encrypted, and i don't know how decrypt without a API request
Regards,
Related
Overview
I tried creating a VPC network, having a subnet and adding a Serverless VPC connector with terraform in GCP. I was following the official guide ( https://cloud.google.com/vpc/docs/configure-serverless-vpc-access#terraform ) and initially everything was working well. After that I accidently commited my JSON key to github, someone stole it and used it for crypto, the project was disabled but shortly after that reinstated
After that my terraform VPC connector creations started to fail. I tried a lot of different things but nothing seems to work(running destroy, changine service accounts, changing names, deleting all of the terraform subfolders, deleting EVERY resource and restarting the process)
The errors I am getting are:
│ Error: Error waiting to create Connector: Error waiting for Creating Connector: Error code 13, message: An internal error occurred: Failed to create a VPC Access connector. Please delete the connector manually.
│
or
│ Error: Error creating Connector: googleapi: Error 409: Requested entity already exists
Today I tried to create VPC connector from the command line(gcloud) and from the UI tool. The errors persisted
Unknown error. Original error message: Operation failed: Insufficient CPU quota in region.
Max throughput of the connector per day over last seven days.
or
An internal error occurred: Failed to create a VPC Access connector. Please delete the connector manually.
errors while deleting:
│ Error: Error waiting for Deleting Network: The network resource 'projects/static-emblem-327016/global/networks/sun-serverless-network' is already being used by 'projects/static-emblem-327016/global/routes/default-route-5cbc9de02e21bb35'
│
I was lookint at this issue https://issuetracker.google.com/issues/164378672 In it I was problems with us-central1 but I tried a couple of different regions and still I have the same issue
Questions:
I am running out of ideas, I was wondering if this is an infrastructural issue, maybe I should dump the project and create a new one ? Where can I check if there are infra issues ? How can I resolve my issue?
I recently get this error Error: Error creating Connector: googleapi: Error 409: Requested entity already exists. So I can explain the root cause and it's fix.
What I was doing is like trying to create a GCP resource (Create PubSub topic) using terraform (plan and then apply).
But before executing the terraform apply, I created the resource manually long time back with the same name. I expected that the terraform plan or terraform apply will not try to create it again since the resource name is same. But instead of Refreshing state, I found it was trying to Creating the resource. The reason it that, terraform does not know about your resource history. Either you need to import your resource history using terraform import command or else delete the manually created resource and then run the terraform apply command.
The message “An internal error occurred: Failed to create a VPC Access connector. Please delete the connector manually” can indicate that you don't have enough resources in your project to create the connector. Please make sure you have enough Resource Quota available in your GCP project.
The message “googleapi: Error 409: Requested entity already exists” indicates that The resource that a client tried to create already exists.
If you want to know what the root cause is, you can check the logs of the VPC Connector creation in the System Event Audit Logs.
System Event audit logs contain log entries for Google Cloud actions that modify the configuration of resources. System Event audit logs are generated by Google systems; they aren't driven by direct user action. System Event audit logs are always written; you can't configure, exclude, or disable them. The instructions to access them are here.
On the other hand, generating and distributing service account keys poses severe security risks to your organization. They are long-lived credentials that are not automatically rotated. These keys can be leaked accidentally or maliciously allow attackers to gain access to your sensitive GCP resources. If you accidentally compromised your JSON Key, please read the recommendations in this link.
If you want to know more about the risk and alternatives to download Service Account, Key please follow this link. Please note that this is not GCP official documentation, so I cannot vouch for its accuracy.
I was able to resolve my issue. It turns out that I had deleted my default compute engine service account in panic. I was able to recover it and everything worked out from there. For more info go here: https://cloud.google.com/iam/docs/creating-managing-service-accounts#undeleting_a_service_account
you have to identify the default service acc for compute engine and undelete it:
gcloud beta iam service-accounts undelete ACCOUNT_ID
I am experiencing database connection errors with an ASP.NET application written in VB, running on three IIS servers. The underlying database is MS Access, which is on a shared network device. It uses Entity Framework, code first implementation and JetEntityFrameworkProvider.
The application is running stable. But, approximately 1 out of 1000 attempts to open the database connection fails with either one of the following two errors:
06:33:50 DbContext "Failed to open connection at 2/12/2020 6:33:50 AM +00:00 with error:
Cannot open database ''. It may not be a database that your application recognizes, or the file may be corrupt.
Or
14:04:39 DbContext "Failed to open connection at 2/13/2020 2:04:39 PM +00:00 with error:
Could not use ''; file already in use.
One second later, with refreshing (F5), the error is gone and it works again.
Details about the environment and used code.
Connection String
<add name="DbContext" connectionString="Provider=Microsoft.Jet.OLEDB.4.0;Data Source=x:\thedatabase.mdb;Jet OLEDB:Database Password=xx;OLE DB Services=-4;" providerName="JetEntityFrameworkProvider" />
DbContext management
The application uses public property to access DbContext. DbContext is kept in the HttpContext.Current.Items collection for the lifetime of the request, and is disposed at it’s end.
Public Shared ReadOnly Property Instance() As DbContext
Get
SyncLock obj
If Not HttpContext.Current.Items.Contains("DbContext") Then
HttpContext.Current.Items.Item("DbContext") = New DbContext()
End If
Return HttpContext.Current.Items.Item("DbContext")
End SyncLock
End Get
End Property
BasePage inits and disposes the DbContext.
Protected Overrides Sub OnInit(e As EventArgs)
MyBase.OnInit(e)
DbContext = Data.DbContext.Instance
...
End Sub
Protected Overrides Sub OnUnload(e As EventArgs)
MyBase.OnUnload(e)
If DbContext IsNot Nothing Then DbContext.Dispose()
End Sub
What I have tried
Many of the questions on SO which address above error messages, deal with generally not being able to establish a connection to the database – they can’t connect at all. That’s different with this case. Connection works 99,99% of the time.
Besides that, I have checked:
Permissions: Full access is granted for share where .mdb (database) and .ldb (locking file) resides.
Network connection: there are no connection issues to the shared device; it’s a Gigabit LAN connection
Maximum number of 255 concurrent connections is not reached
Maximum size of database not exceeded (db has only 5 MB)
Changed the compile option from “Any CPU” to “x86” as suggested in this MS Dev-Net post
Quote: I was getting the same "Cannot open database ''" error, but completely randomly (it seemed). The MDB file was less than 1Mb, so no issue with a 2Gb limit as mentioned a lot with this error.
It worked 100% on 32 bit versions of windows, but I discovered that the issues were on 64 bit installations.
The app was being compiled as "Any CPU".
I changed the compile option from "Any CPU" to "x86" and the problem has disappeared.
Nothing helped so far.
To gather more information, I attached an Nlog logger to the DbContext which writes all database actions and queries to a log file.
Shared Log As Logger = LogManager.GetLogger("DbContext")
Me.Database.Log = Sub(s) Log.Debug(s)
Investigating the logs I figured out that when one of the above errors occured on one server, another one of the servers (3 in total) has closed the db connection at exactly the same time.
Here two examples which correspond to the above errors:
06:33:50 DbContext "Closed connection at 2/12/2020 6:33:50 AM +00:00
14:04:39 DbContext "Closed connection at 2/13/2020 2:04:39 PM +00:00
Assumption
When all connections of a DbContext have been closed, the according record is removed from the .ldb lock file. When a connection to the db is being opened, a record will be added to the lock file. When these two events occur at the exact same time, from two different servers, there is a write conflict to the .ldb lock file, which results in on of the errors from above.
Question
Can anyone confirm or prove this wrong? Has anyone experienced this behaviour? Maybe I am missing something else. I’d appreciate your input and experience on this.
If my assumption is true, a solution could be to use a helper class for accessing db, which catches and handles this error, waiting for a minimal time period and trying again.
But this feels kind of wrong. So I am also open to suggestions for a “proper” solution.
EDIT: The "proper" solution would be using a DBMS Server (as stated in the comments below). I'm aware of this. For now, I have to deal with this design mistake without being responsible for it. Also, I can't change it in the short run.
I write this as an aswer because of space but this is not really an answer.
It's for sure an OleDb provider issue.
I think that is a sharing issue.
You could do some tries:
use a newer OleDb provider instead of Microsoft.Jet.OLEDB.4.0. (if you have try 64 bits you could already have try another provider because Jet.OLEDB.4.0 is 32 bits only)
Implement a retry mechanism on the new DbContext()
Reading your tests this is probaly not your case. I THINK that Dispose does not always work properly on Jet.OLEDB.4.0 connections. I noted it on tests and I solved it using a different testing engine. Before giving up I used this piece of code
GC.Collect(GC.MaxGeneration, GCCollectionMode.Forced, true);
GC.WaitForPendingFinalizers();
GC.Collect(GC.MaxGeneration, GCCollectionMode.Forced, true);
As you can understand reading this code, they are tries and the latest solution was changing the testing engine.
If your app is not too busy you could try to lock the db using a different mechanism (for example using a lock file). This is not really different from new DbContext() retries.
In late '90s I remember I had an issue related to disk sharing OS (I were using Novel Netware). Actually I have not experience in using mdb files on a network share. You could try to move the mdb on a folder shared with Windows
Actually I use Access databases only for tests. If you really need to use a single file database you could try other solutions: SQL Lite (you need a library, also this written by me, to apply code first https://www.nuget.org/packages/System.Data.SQLite.EF6.Migrations/ ) or SQL Server CE
Use a DBMS Server. This is for sure the best solution. As the writer of JetEntityFrameworkProvider I think that single file databases are great for single user apps (for this apps I suggest SQL Lite), for tests (I think that for tests JetEntityFrameworkProvider is great), for transfering data or, also, for readonly applications. In other cases use a DBMS Server. As you know, with EF, you can change from JetEntityFrameworkProvider to SQL Server or to MySql without effort.
You went wrong at the design stage: The MS Access database engine is unfit for ASP.Net sites, and this is explicitly stated on multiple places, e.g. the official download page under details.
The Access Database Engine 2016 Redistributable is not intended .... To be used by ... a program called from server-side web application such as ASP.NET
If you really have to work with an Access database, you can run a helper class that retries in case of common errors. But I don't recommend it.
The proper solution here is using a different RDBMS which exhibits stateless behavior. I recommend SQL Server Express, which has limitations, but if you exceed those you will be far beyond what Access supports, and wont cause errors like this.
I support an application who call a CMD line to decrypt a file.
The application is a .exe file that is called by the Windows Task Scheduler and is execute as the same user who have all right.
The application run every week day in the evening at 6h30pm and sometimes the CMD line return the message: no secret key.
The application failed because the file was not decrypted. But it doesn't failed every evening, just random evening. It looks totally random.
And if I run the application myself after it failed with the same user, it worked.
The secret key is imported in Kleopatra and it work fine with other application that run in the morning. And it work fine when I used it.
What can cause this?
Thank you
We fix the problem. We must not log off the application user.
If we log off the user, one key is not working, but the others are working.
Some ideas to help you run down the problem:
Check the private keys available to the machine on which the application fails
gpg --list-secret-keys
(IIRC Kleopatra runs on top of GnuPG, so I assume your application does as well. I've been wrong before.) You might notice something out of place with your private (decryption) keys. For example, if the key is listed as either
sec#
ssb>
Then it's a (primary or sub respectively) key located on a smart card for storage. If the card, for whatever reason, isn't in the machine when the app runs it'll fail to decrypt.
Check the disk containing the private keyring is attached/inserted/mounted at the time the application ran and failed to decrypt. If the keys are stored on removable (or unreliable) media then that could also result in a failure to decrypt.
Check that the item failing to decrypt was encrypted properly. If there is some secondary recipient necessary for the app to run there may be a required key that you don't know about (I gather from your post you didn't create this app, just maintain it.) It may even be that the app is trying to decrypt a different file erroneously, but that kind of thing can only be found out by stepping through your source code and resident files.
Failing those, pray for #Jens Erat to notice your question.
On a fresh Centos server I've installed Riak CS 2.1.1 on top of Riak 2.1.3 with Stanchion 2.1.1 using official guide.
I was able to create new users when anonymous_user_creation is enabled in Riak CS conf file. But when I disable it and try to create new user via Riak Control (1.0.2) I get an 409 Conflict HTTP status code. Or when I try to list all users via s3curl (I've set up ~/.s3curl file with right credentials and used ./s3curl.pl --id=cdn -- -k https://s3.example.com:8081/riak-cs/users command to send request) I get an error:
<?xml version="1.0" encoding="UTF-8"?><Error><Code>AccessDenied</Code><Message>Access Denied</Message><Resource>/riak-cs/users</Resource><RequestId></RequestId></Error>
I've set admin.key and admin.secret in both Riak CS and Stanchion configs to values that I got creating anonymous user. I tried to create another anonymous user and double checked that I'm taking right values for key and secret.
There is no trace of errors in Riak, Stanchion or Riak CS logs - except Riak CS access logs, but that doesn't tell much.
I have no idea what to do next. Any suggestions would be welcome.
Thanks to #Joe help, I was able to debug this issue at least for s3curl.
It turned out to be related to the fact that I've chosen "direct" configuration of Riak CS instead of "proxy". That means that I have given it its own hostname instead of "s3.amazonaws.com" (Riak CS is really trying to disguise as real S3 due to clients compatibility issues).
And while Riak CS used this string to calculate checksum (in erlang-y format):
["GET","\n",[],"\n",[],"\n",["Mon, 08 Feb 2016 14:00:58 +0000","\n"],[],["/riak-cs/users",[]]]
s3curl, having encountered unknown hostname, used this one:
GET\n\n\nMon, 08 Feb 2016 14:10:28 +0000\n/s3.mydoma.in/riak-cs/users
Which caused Access Denied.
Fix - add to ~/.s3curl string like:
push #endpoints, 's3.mydoma.in';
Now I'll try to debug Riak CS Control.
During vm creation in openstack, one can specify a keypair name, so that the specified public key get injected to the newly created vm.
I would like to know in which state of machine the key injection is done, completely? Given the machine is in ACTIVE state, does that guarantee that the key injection is completed?
Details:
I have a limited quota for the key pairs and I would like to delete each keypair from openstack immediately after they get injected to the target machine. I have only access to openstack ReST API and NOT to the target vm.
UPDATE
Looking at nova instances table, I can see that "key name" and "key data" are existing there too. I think the key is copied to this table and then the original key is not referenced any more. So deleting the key shouldn't cause any issue. am I wrong?
What you can do is try a ssh connection and once that succeeds, proceed to delete the keypair.
To answer your question directly, the key is added via the cloud-init. You can grep for ssh in /var/log/cloud-init.log to see when exactly it happens. (It happens pretty early in the cloud-init process).
I don't think there is any API way of figuring out when exactly the key injection happens. Machine in ACTIVE state is not a guarantee that cloud-init part of key injection is done (though for practical purposes, it does happen pretty early).
You could try checking it via nova console-log. Though the output of console-log has limited buffer, so it may overshoot the key addition part and hence you may not see it in console log.
So, I think checking via actual ssh connection is the only sure shot way.