Configure Fluent-bit file output plugin set file rollover - fluent-bit

I've a Fluent-bit running as docker instance which uses tail input plugin to read docker log files and currently I've configured output plugin to use Elasticsearch.
However in prod since Elasticsearch resides on another zone, don't want to consume bandwidth I don't want to use Elasticsearch, instead want to use file output plugin. But in file plugin there is no documentation explaining the file rollover configuration.

At the moment there is no such feature in Fluent Bit File output plugin. Please log the enhancement request on GitHub:
https://github.com/fluent/fluent-bit/issues/new?template=feature_request.md
Since this feature makes a lot of sense it can be implemented shortly,
regards,
Eduardo
Fluent Bit Maintainer

Related

Writing an appspec.yml File for Deployment from S3 (and/or Bit Bucket) to AWS CodeDeploy

I'd like to make it so that a commit to our BitBucket repo (or S3 Bucket) automatically deploys code (using CodeDeploy) to our EC2 instances. I'm not clear what to use for the 'source' and 'destination' entry under the 'files' section in the appspec.yml file and also I am not cleared what to mention in BeforeInstall and AfterInstall under 'Hooks' section. I've found some examples on Google and AWs documentation but I am confused what to mention in above fields. The more I am exploring more I am getting confused.
Consider I am new to AWS Code Deploy.
Also it will be very helpful if someone can provide me step y step link how to configure and how to automate the CodeDeploy.
I was wondering if someone could help me out?
Thanks in advance for your help!
Thanks for using CodeDeploy. For new users, I'd like to recommend the following things to do:
Try to run First Run Wizard on console, it will should you the general process how the deployment goes. It also provide a default deployment bundle, also an appspec file included.
Once you want to try a deployment yourself, the Get Started doc is a great place to help you with some pre-requiste settings like IAM role
Then probably try some tutorials for a sample app too, which gives you some idea about deployment groups, deployment configuration, revision and so on.
The next step should be create a bundle for your own use cases, Appspec file doc would be a great place to refer. And for your concerns about BeforeInstall and AfterInstall, if your application doesn't need to do anything, the lifecycle events can be left as empty. BeforeInstall can be used to for for preinstall tasks, such as decrypting files and creating a backup of the current version, while AfterInstall can be used for tasks such as configuring your application or changing file permissions.
Now it comes to the fun part! This blog talks about details about how to integrate with Github(similar for Bitbucket). It's a little long, but really useful, and it also includes how to do automatically deployment once there is a new pushed commit. Currently Jenkins and CodePipline are really popular for auto-triggered deplyoments, but there are always a lot of other ways can achieve the same purpose like Lamda and so on

How to edit source code in local computer and server simultaneously

I am a new web developer. I am using XAMPP and Dreamweaver CS6. I want to edit and debug my source code on my local computer. Then I want to commit that edit to my web server (shared or vps). Right now I edit files on local computer and then I upload that file to server.
Can anyone suggest any other options.
Thank you in advance.
I suppose you could achieve this in many ways.
For example you can use deployment tools (like Jenkins) and then configure it to hook your commits. In order to do this you need first to install Git in your system (I'm not sure but maybe you can even find some Git plugins for Dreamweaver so that you could work with Git directly from Dreamweaver; if not don't worry there are plenty of Git clients that help to facilitate use of Git, SourceTree is one of them)
If you feel like you have difficulty in configuring Jenkins to hook your commits I'd suggest you to have a look at this answer.
I haven't used Dreamweaver in forever but you can upload on saving which you set in the Site Definition. Go to the Remote definition that you setup and click on the Advanced tab. You can set DW to automatically upload on save there. It's generally not best practice but it's an option.

How to deploy my Symfony2 project into ftp

I searched and tried a couple of tutorials on the internet but none of them worked for me well.
The tutorials I followed were from Symfony2 documentation, Dator, Hpatoio and Capifony.
Can somebody explain to me how I can export my project into my server. E.g. www.domain.com/about.
It would be very helpful to me.
I have a bundle and inside the bundles all controller and twig templates etc are set.
If you have any questions please ask.
Thanks in advance.
First off it should be noted that deploying a Symfony2 app over FTP is really really bad. It makes a couple of steps more difficult (or even impossible) and should be avoided. If you have SSH access to the machine look at my list of alternative deployment methods below.
Preparation
There are a few things you cannot influence when you deploy over FTP. If you have no control over the following or can not configure them correctly you unfortunately have no chance of deploying to a shared hosting.
The PHP configuration. If settings are not set correctly and you have no chance of changing them you are unfortunately lost.
Any PHP module you may require. Same as above. If you can not install any additional modules you need there is no chance for you. (An example for that would be the php5-intl module for any Symfony <2.6)
Correct folder permissions. Especially for app/cache and app/logs. Check the docs for requirements.
The Webserver configuration. Symfony needs a correctly configured Webserver (in most cases probably apache or nginx) to function correctly. If you can not influence the config this is bad as well. You might want to try to define the rewrite rules in a .htaccess file as described here.
Deployment
Here are the steps you need to follow to prepare your application for deployment for the first time:
Update / Install vendors. Use composer install (or composer update) to install any third party bundle or library you use in your project since you have no option to install them later directly on the server.
If you use Assetic for your asset management, make sure you install these as well with the php app/console assetic:dump --env=prod command.
Dump any other assets like so: php app/console assets:install --env=prod. (This step might not be required but you need to make sure the assets are not symlinked. Check this blog post if you are using symfony >=2.6)
Clear the cache for production: php app/console cache:clear --env=prod
Make sure you edit your parameters.yml to fit the needs of your production server.
Also update your database schema on your production database in case you have changed it during development.
Now you should be good to go. Copy the whole folder onto your server and try it out.
For the future
If you deploy for the second time make sure not to override any user data (e.g. uploaded images). Also you need to clear the cache over ftp. To do that empty the app/cache directory.
Alternative deployment methods
In case you have more access to the server check out any of these. They might fit your needs better than old pure FTP. Maybe they give you reason enough to change to a more appropriate server. Capifony is probably one of the best deployment tools for Symfony2 apps. Deployment will be as easy as running cap deploy on your local machine. The rest is magic ;) Simple git is also possible for deployment. Many of the above steps will still apply but you have all the advantages git gives you like not copying everything every time you deploy. A very good list of all tools can be found in the docs.
It might help if you tell us a little more about your server set up, but here's a fairly generic guide:
Assuming you want to upload it using ftp (since you tagged the question as such), you will need an FTP
client (see here for some suggestions).
Using the FTP client, you'll want to connect to your server
(hostname: yoursite.com) using your credentials (if it is a secured
server).
From there, you should be able to upload any files from your local
machine to the server.
More specific directions will depend on your server configuration and the FTP client you choose (it should come with its own manual)

Install WordPress with its plugins using chef-solo

I have a WordPress website up and running with many plugins installed on it and a huge database, I need to use chef-solo in order to create an environment in which can install the same website with all its plugins and and also importing its database.
I need it to be like, using chef to install the same website on a different server, exactly the same
Now here are my questions:
I know we can use chef to install WordPress but can we set it in a
way that we don't need to configure the the WordPress and everything
is already set once its running?
What to do with the plugins? can we install them using the chef or
now that should be done manually?
How about importing the database, that can be done with chef-solo
as well?
The whole website is on git, can I somehow import the whole
thing?
is there any other issue I may possibly face? if I want do that?
There is a wordpress cookbook openly available for chef.
When you mean configure, I take it you mean setup data in the database. Assuming that you've separated the database instance from the server instance, and you're attempting to scale up the number of servers then you should be able to skip data setup. You should be configuring the new server instance (node) to point to the same database via Chef.
I stumbled into this question looking for the answer to this question. From what I can tell the start may be here.
Kind of hand-wavy, but this should enable you to do some wordpress stuff via the command line with Chef, rather than the point and click it prefers.
As per #1, you should not need to import the database. If the database goes down, you'll want to focus on that as a separate but connected recipe, since then you'll want to be taking snapshots and uploading them somewhere like S3 via a cron job. I believe there are plugins that can enable this.
You'll have to be a little more clear by "import". If it's in a code base you may be able to short-cut your cookbook path by pulling down the git repo onto the host. You may want to look at git-archive.
Other issues that I'm looking at are images. We're migrating from a hosted solution to AWS, and it appears that instead of storing the images in the database, word-press pulls them into a local directory. This means that if we scale to > 1 host, we'll have issues with images. Something to think about, there's a wealth of plugins that can probably solve this.
Hope this is helpful,
Ben

How can I upload to Confluence using a unix curl command?

OK, so I know how to upload to SharePoint thanks to this question:
How to send file to Sharepoint from Linux creating non existend directories
Now I am trying to figure out how to do it with Atlassian's Confluence. Any takers?
What I am looking for is a scriptable Unix command.
You question needs improving for a real understanding of what you're trying to achieve. That said, you can for example attach a file to a page within Confluence using curl.
The easy way to use do this would be using Bob Swifts Atlassian/Confluence CLI using the addAttachment command to add an attachment.
The plugin changed to be commercial a while back, but you can download the latest free release here.

Resources