Currently i have an OpenStack environment with Ceph as a backend storage driver for Cinder.
I have looked into cinder documentation and code and i could not find any values or options to set the default stripe_unit or stripe_count for rbd volumes.
The reason i want to do this is that i want to have my volumes striped.
Is it possible to set default stripe_count and stripe_unit in ceph.conf ?
i have tried to add the following to the [client] section in ceph.conf , but it did not work
rbd stripe-count N
rbd stripe-unit N
Any advise ?
Needed to set these values in the ceph.conf file :
I also needed to enable some features for expirements and i needed to increase the object size as well, this is what order is.
rbd default features = 12
rbd default format = 2
rbd default stripe_count = 16
rbd default stripe_unit = 4194304
rbd default order = 23
Related
Using: telegraf version 1.23.1
Thats the workflow Telegraf => Influx => Grafana.
I am using telegraf to check my metrics on a shared server. So far so good, i already could initalize the Telegraf uWSGI Plugin and display the data of my running django projects in grafana.
Problem
Now i wanted to check some folder size too with the [[inputs.filecount]] Telegraf Plugin and this works also well. However i do not need Metrics for every 10s for this plugin. So i change the interval like mentioned in the Documentation in the [[inputs.filecount]] Plugin.
telegraf.conf
[agent]
interval = "10s"
round_interval = true
metric_batch_size = 1000
metric_buffer_limit = 10000
collection_jitter = "5s"
flush_interval = "10s"
flush_jitter = "0s"
#... PLUGIN
[[inputs.filecount]]
# set different interval for this input plugin every 10min
interval=“600s”
collection_jitter=“20s”
# Default from Doc =>
directories = ["/home/myserver/logs", "/home/someName/growingData, ]
name = "*"
recursive = true
regular_only = false
follow_symlinks = false
size = "0B"
mtime = "0s"
After restarting Telegram with Supervisor it crashed because it could not parse the new lines.
supervisor.log
Error running agent: Error loading config file /home/user/etc/telegraf/telegraf.conf: Error parsing data: line 208: invalid TOML syntax
So that are these lines i added because i thought that is how the Doc it mention it.
telegraf.conf
# set different interval for this input plugin every 10min
interval=“600s”
collection_jitter=“20s”
Question
So my question is. How can i change or setup the interval for a single input plugin in telegraf?
Or do i have to apply a different TOML syntax like [[inputs.filecount.agent]] or so?
I assume that i do not have to change any output interval also? Because i assume even though its currently 10s, if this input plugin only pulls/inputs data every 600s it should not matter, some flush cycle will push the Data to influx .
How can i change or setup the interval for a single input plugin in telegraf?
As the link you pointed to shows, individual inputs can set the interval and collection_jitter options. There is no difference in the TOML syntax for example I can do the following for the memory input plugin:
[[inputs.mem]]
interval="600s"
collection_jitter="20s"
I assume that i do not have to change any output interval also?
Correct, these are independent of each other.
line 208: invalid TOML syntax
Knowing what exactly is on line 208 and around that line will hopefully resolve your issue and get you going again. Also make sure your quotes that you used are correct. Sometimes when people copy and paste quotes they get ” vs " which can cause issues!
I'm following the wiki https://help.ubuntu.com/community/Xen#Manually_Create_a_PV_Guest_VM
(section "
Set Up Initial Guest Configuration
")
I downloaded the netboot initrd.gz from https://mirror.arizona.edu/ubuntu//ubuntu/dists/bionic/main/installer-amd64/current/images/netboot/xen/
but in the .cfg , what should I specify for the "disk = " line? ---- my host box is not using LVM, so I'll have to use "file-backed storage" for PV disk image. (https://wiki.xenproject.org/wiki/Storage_options , indeed this worked when I gave --dir= instead of --lvm= when running the xml-create command in https://wiki.xenproject.org/wiki/Xen_Project_Beginners_Guide )
here is my current config:
yy#yy-70A4000HUX:~/ub_xen$ cat ub_xen.cfg
name = "ubud1"
kernel = "/home/yy/ub_xen/vmlinuz"
ramdisk = "/home/yy/ub_xen/initrd.gz"
#bootloader = "/usr/lib/xen-4.4/bin/pygrub"
memory = 1024
vcpus = 1
# Custom option for Open vSwitch
vif = [ 'bridge=xenbr0' ]
disk = [ 'vdev=hda,target=/home/yy/ub_xen/images' ]
# You may also consider some other options
# [[http://xenbits.xen.org/docs/4.4-testing/man/xl.cfg.5.html]]
yy#yy-70A4000HUX:~/ub_xen$
I ran the command with sudo xl create -c ub_xen.cfg
this worked fine first, giving me the regular install process on console, pulling install files from remote archive, but when it comes to the step of disk paritioning, it's showing me a "SCSI" partitioning choice, with no volumes / partitions/disks to be chosen.
I guess this is because I'm not setting the right value for "disk = [ ]" option. what should I use here if I use file-backed storage for PV (just like VMware does)?
thanks a lot
Yang
found it, huge thanks to the author here: https://www.systutorials.com/create-and-manage-virtual-machines-on-xen/
When creating a PartitionSetDefinition in Dagster you can pass in a 'mode' that will swap the resources used (for testing purposes you may want to use cloud storage in PROD but use local storage for local development
A mode requires you to specify a set of config values that are usually provided in an environment yaml file but when you create a PartitionSetDefinition like below you can only pass the mode. This is usually done by setting a preset on the pipeline and using that for the run but PartitionSetDefinition only allows the setting of a mode not a preset.
date_partition_set = PartitionSetDefinition(
name="date_partition_set",
pipeline_name="my_pipeline",
partition_fn=get_date_partitions,
run_config_fn_for_partition=run_config_for_date_partition,
mode="test"
)
How can you provide the necessary preset/environment values for this?
One way I've found to do this is to load the presets into the run config when the run config is created for each partition using some utilities that Dagster provides. Found this in some of their unit tests:
test_base.yaml has the typical preset configs corresponding to test mode.
from dagster.utils import file_relative_path, load_yaml_from_globs
def run_config_for_date_partition(partition):
date = partition.value
config_path = file_relative_path(__file__, os.path.join("../my_pkg/environments/", relative_path))
config_dict = load_yaml_from_globs(
config_path("test_base.yaml"),
)
table_name = "table1"
input_config = {"config": {"start_date": date, "table_name": table_name}}
config_dict["solids"] = {
"download_snow_incremental_table": {**input_config}
}
return config_dict
date_partition_set = PartitionSetDefinition(
name="date_partition_set",
pipeline_name="my_pipeline",
partition_fn=get_date_partitions,
run_config_fn_for_partition=run_config_for_date_partition,
mode="test"
)
I have a requirement to setup CDC from Source(Oracle) to Target(BigQuery) using Goldengate.
I can have only option to filter data in replicat side based on Specific column name .
As per the below link :
https://docs.oracle.com/en/cloud/paas/goldengate-cloud/gwuad/using-oracle-goldengate-parameter-files.html#GUID-7F405A81-B2D1-4072-B254-DC2B0EC56FBA
I have setup the replicat like below
REPLICAT RPOC
TARGETDB LIBFILE libggjava.so SET property=dirprm/bqpoc.props
SOURCEDEFS /app/oracle/ogg_bigdata/dirdef/poc.def
REPORTCOUNT EVERY 1 MINUTES, RATE
GROUPTRANSOPS 500
MAP ARADMINPI.TPOC, TARGET PRD.TPOCFL,KEYCOLS(ID),WHERE (NAME= ?SOUVIKPOC);
===================================
export SOUVIKPOC='Smith'
But I am getting below error
2020-02-19 05:47:37 ERROR OGG-01157 Error in WHERE clause for ARADMINPI.TPOC.
=============================
Is there anything I am doing wrong here?
For Parameter Substitution to work, you'll need to enclose ?SOUVIKPOC in quotes, like this:
MAP ARADMINPI.TPOC, TARGET PRD.TPOCFL,KEYCOLS(ID),WHERE (NAME= '?SOUVIKPOC');
There should also be additional information about the failure earlier in the report file.
Another Example Using #GETENV
Another option is to use the #GETENV function instead of Parameter Substitution. Here, the MAP statement uses a FILTER clause instead of the WHERE clause:
MAP ARADMINPI.TPOC, TARGET PRD.TPOCFL,KEYCOLS(ID),
FILTER (#STREQ(NAME, #GETENV('OSVARIABLE', 'SOUVIKPOC')));
Unless you set the SOUVIKPOC environment variable prior to running GGSCI (and executing START MGR), you need to add a SETENV statement to your parameter file:
SETENV (SOUVIKPOC = 'Smith')
Putting it all together:
REPLICAT RPOC
TARGETDB LIBFILE libggjava.so SET property=dirprm/bqpoc.props
SOURCEDEFS /app/oracle/ogg_bigdata/dirdef/poc.def
REPORTCOUNT EVERY 1 MINUTES, RATE
GROUPTRANSOPS 500
SETENV (SOUVIKPOC = 'Smith')
MAP ARADMINPI.TPOC, TARGET PRD.TPOCFL,KEYCOLS(ID),
FILTER (#STREQ(NAME, #GETENV('OSVARIABLE', 'SOUVIKPOC')));
we have recently upgraded to oe rdbms 11.3 version from 9.1d. While generating
reports,i found the field value of a field comes as 2'239,00 instead of
2,239.00.I checked the format its >,>>>,>>9.99.
what could be the reason behind this?
The admin installing the database didn't to it's homework and selected wrong default numeric and decimal separator.
However no greater harm done:
Set these startup parameters
-numsep 44 -numdec 46
This is an simplified database startup example with added parameters as above:
proserve /db/db -H dbserver -S dbservice -numsep 44 -numdec 46
When you install Progress you are prompted for the numeric format to use. That information is then written to a file called "startup.pf" which is located in the install directory (C:\Progress\OpenEdge by default on Windows...)
If you picked the wrong numeric format you can edit startup.pf with any text editor. It should look something like this:
#This is a placeholder startup.pf
#You may put any global startup parameters you desire
#in this file. They will be used by ALL Progress modules
#including the client, server, utilities, etc.
#
#The file dlc/prolang/locale.pf provides examples of
#settings for the different options that vary internationally.
#
#The directories under dlc/prolang contain examples of
#startup.pf settings appropriate to each region.
#For example, the file dlc/prolang/ger/german.pf shows
#settings that might be used in Germany.
#The file dlc/prolang/ger/geraus.pf gives example settings
#for German-speaking Austrians.
#
#Copy the file that seems appropriate for your region or language
#over this startup.pf. Edit the file to meet your needs.
#
# e.g. UNIX: cp /dlc/prolang/ger/geraus.pf /dlc/startup.pf
# e.g. DOS, WINDOWS: copy \dlc\prolang\ger\geraus.pf \dlc\startup.pf
#
# You may want to include these same settings in /dlc/ade.pf.
#
#If the directory for your region or language does not exist in
#dlc/prolang, please check that you have ordered AND installed the
#International component. The International component provides
#these directories and files.
#
-cpinternal ISO8859-1
-cpstream ISO8859-1
-cpcoll Basic
-cpcase Basic
-d mdy
-numsep 44
-numdec 46
Changes to the startup.pf file are GLOBAL -- they impact all sessions started on this machine. If you only want to change a single session then you can add the parameters to the command line (or the shortcut icons properties) or to a local .pf file or to an ini file being used by that session.
You can also programmatically override the format in your code by using the SESSION system handle:
assign
session:numeric-decimal-point = "."
session:numeric-separator = ","
.
display 123456.999.
(You might want to consider saving the current values and restoring them if this is a temporary change.)
(You can also use the shorthand session:numeric-format = "american". or "european" for the two most common cases.)