According to the picture from this page ceph splits objects on chunks and writes them on osds.
For viewing objects distribution with replication algorithm I can use commands like ceph pg dump or ceph osd map.
But I can't find commands to view data chunk distribution.
You can use the ceph-objectstore-tool to query offline OSDs and see which OSD contains which data chunk:
[ceph: root#pacific /]# ceph-objectstore-tool --data-path /var/lib/ceph/osd/ceph-1/ --op list
["11.2s2",{"oid":"file","key":"","snapid":-2,"hash":779072666,"max":0,"pool":11,"namespace":"","shard_id":2,"max":0}]
The first entry contains the shard ID: "11.2s2". So the PG is 11.2 and shard ID is 2. To query a cephadm deployed OSD you need to stop it via cephadm and then enter the container:
pacific:~ # cephadm unit stop --name osd.1
Inferring fsid 0a8034bc-15f4-11ec-8330-fa163eed040c
pacific:~ # cephadm shell --name osd.1
Related
Why are some openstack instances found in the "dashboard" and viewed using the command 'nova list --ip [ip]', but not found using the command 'openstack server list|grep [ip]', 'nova list|grep [ip]'?
It is hard to say for sure what the cause of your problem is from the information you provided.
However, nova list and openstack server list both default to listing instances for the current project only. To list instances for all projects, you need to include the --all-tenants or --all-projects option respectively.
Refer to the Nova list command command to select parameter parameters:
--limit
Maximum number of servers to display. If limit == -1, all servers will be displayed. If limit is bigger than 'CONF.api.max_limit' option of Nova API, limit 'CONF.api.max_limit' will be used instead.
I connect to snowflake via both odbc and snowsql. For whatever reason, when querying via odbc and looking at running queries in the history tab, the size is set to Large. When querying via SnowSQL the size is set to small.
I did some searching for odbc and snowsql config but could not tell if this is something I set within the UI or via the connection settings in either client.
Can I set size via a connection setting for either odbc or SnowSQL? If yes, how?
E.g. my current sanitized connection settings:
SnowSQL:
Config file:
[connections]
authenticator = SNOWFLAKE_JWT
private_key_path = /root/.snowsql/rsa_key.p8
[variables]
#Loads these variables on startup
#Can be used in SnowSql as select $example_variable
example_variable=27
[options]
# If set to false auto-completion will not occur interactive mode.
auto_completion = True
# going to use start and end date variables from environment in the sql script
variable_substitution = True
# main log file location. The file includes the log from SnowSQL main
# executable.
log_file = /root/.snowsql/snowsql_rt.log
# bootstrap log file location. The file includes the log from SnowSQL bootstrap
# executable.
log_bootstrap_file = /root/.snowsql/log_bootstrap
# Default log level. Possible values: "CRITICAL", "ERROR", "WARNING", "INFO"
# and "DEBUG".
log_level = DEBUG
# Timing of sql statments and table rendering.
timing = True
# Table format. Possible values: psql, plain, simple, grid, fancy_grid, pipe,
# orgtbl, rst, mediawiki, html, latex, latex_booktabs, tsv.
# Recommended: psql, fancy_grid and grid.
output_format = psql
# Keybindings: Possible values: emacs, vi.
# Emacs mode: Ctrl-A is home, Ctrl-E is end. All emacs keybindings are available in the REPL.
# When Vi mode is enabled you can use modal editing features offered by Vi in the REPL.
key_bindings = emacs
# OCSP Fail Open Mode.
# The only OCSP scenario which will lead to connection failure would be OCSP response with a
# revoked status. Any other errors or in the OCSP module will not raise an error.
# ocsp_fail_open = True
# Enable temporary credential file for Linux users
# For Linux users, since there are no OS-key-store, an unsecure temporary credential for SSO can be enabled by this option. The default value for this option is False.
# client_store_temporary_credential = True
# Repository Base URL
# The endpoint to download the SnowSQL main module.
repository_base_url = https://sfc-repo.snowflakecomputing.com/snowsql
I use this config when connecting with the following:
snowsql -f ${INPUT_QUERY_FILE} \
-o quiet=true \
-o friendly=false \
-o header=true \
-o output_format=csv \
-o output_file=output_data/data.csv \
--accountname=${INPUT_ACCOUNT_NAME} \
--username=${INPUT_USER_NAME} \
--dbname=${INPUT_DBNAME} \
--private-key-path=/root/.snowsql/rsa_key.p8 \
--config /config
This runs as expected and returns the query result. But it uses a small setting when querying Snowflake. Is there a way to adjust this, e.g. large?
And for odbc:
[snowflake]
Description=SnowflakeDB
Driver=SnowflakeDSIIDriver
Locale=en-US
SERVER=<ourorg>.us-east-1.snowflakecomputing.com
PORT=443
SSL=on
ACCOUNT=<ourorg>.us-east-1
# change to your snowflake user_name
UID=MY_NAME
# change to /home/<your-home>/keys/rsa_key.p8
PRIV_KEY_FILE=/home/my_name/keys/rsa_key.p8
# change "blahblah' to the passphrase you set for your private key in step 1.c
PRIV_KEY_FILE_PWD=blahblah
AUTHENTICATOR=SNOWFLAKE_JWT
And then I connect with (R)
sfconn <- DBI::dbConnect(odbc::odbc(), dsn = 'snowflake', warehouse = 'DATA_SCIENCE_WH_L',
database = 'ourorg', role='DATA_SCIENCE_FULL')
This connects fine and I'm able to query Snowflake using odbc client. The query is run with the 'large' settings. Is it possible to adjust this?
By 'size' do you mean the virtual warehouse size? Are you specifying the warehouse when connecting via SnowSQL or defaulting to one that is sized as small? I see on the ODBC line that you are using a warehouse name of DATA_SCIENCE_WH_L while you are not listing a warehouse (or role) on the SnowSQL connection string.
I have a 3-node Galera MariaDB cluster and I want to have a supplementary backup using mysqldump for restore of individual tables in the event of any user errors. Currently Node1 is being used by all applications while node2 and node3 are just kept in sync. I want to run mysqldump from idle Node3. Should I not use --flush-logs? Also should I use --master-data option?
I ran mysqldump backup in a pre-prod cluster (same setup as production) from an idle node Node3 with these options
But as soon as I ran mysqldump, the data in few tables (checked only few at random) and they were not in sync with other nodes. But in few minutes it came back in sync with other nodes.
mysqldump -u root -pPassword --host=localhost --all-databases --flush-logs --events --routines --single-transaction --master-data=2 --include-master-host-port
My question is:
a) Should I avoid using --flush-logs option in my mysqldump? --Is it the cause for the current node going out of sync?
b) Should I even include --master-data option in the mysqldump command?
Take node3 out of the cluster.
Do whatever dump you like (mysqldump, copy disk, xtrabackup, etc)
Put back into the cluster -- it will repair itself to get back in sync.
I have small example script (script_p.r) like the following, which in intend run in terminal.
#!/usr/bin/Rscript
sink("output_capture.txt")
mn <- mean(1:10)
# and so on, much longer list of tasks
I want to run this script remotely with other iMac host computer (ip address e.g.. not real : 111.111.111.111) which allows me to log in and work (e.g., not real. username user101, password p12334)
Is this way to run this script remotely (say using ssh), say from other computer with ip address: 222.222.222.222 and user name user102 ?
First, put script_p.r on the remote machine.
Then either just do:
ssh user102#222.222.222.222
user102:-$ ./script_p.r
or ssh user102#222.222.222.222 'script_p.r'
or put it in a script:
runremote.sh :
#!/bin/bash
ssh user102#222.222.222.222 'script_p.r'
and run locally
user101:-$ ./runremote.sh
I used this code from the command prompt on a windows box (linux machine is at work):
ftp -u ftp://cran.R-project.org/incoming/ qdap_0.1.0.tar.gz
I used the info from:
https://github.com/hadley/devtools/wiki/Release
http://cran.r-project.org/doc/manuals/R-exts.html#Submitting-a-package-to-CRAN
I expected to see it show up here: ftp://cran.r-project.org/incoming/ but I do not see it.
Am I just being impatient or did my package not upload? Here is the command line output:
C:\Users\trinker\GitHub>ftp -u ftp://cran.R-project.org/incoming/ qdap_0.1.0.tar
.gz
Transfers files to and from a computer running an FTP server service
(sometimes called a daemon). Ftp can be used interactively.
FTP [-v] [-d] [-i] [-n] [-g] [-s:filename] [-a] [-A] [-x:sendbuffer] [-r:recvbuf
fer] [-b:asyncbuffers] [-w:windowsize] [host]
-v Suppresses display of remote server responses.
-n Suppresses auto-login upon initial connection.
-i Turns off interactive prompting during multiple file
transfers.
-d Enables debugging.
-g Disables filename globbing (see GLOB command).
-s:filename Specifies a text file containing FTP commands; the
commands will automatically run after FTP starts.
-a Use any local interface when binding data connection.
-A login as anonymous.
-x:send sockbuf Overrides the default SO_SNDBUF size of 8192.
-r:recv sockbuf Overrides the default SO_RCVBUF size of 8192.
-b:async count Overrides the default async count of 3
-w:windowsize Overrides the default transfer buffer size of 65535.
host Specifies the host name or IP address of the remote
host to connect to.
Notes:
- mget and mput commands take y/n/q for yes/no/quit.
- Use Control-C to abort commands.
(This was previously a comment and is being transferred to an answer here.)
Make sure you are not looking at a page cached earlier by your browser.
To perform the actual upload you might want to try the free cross platform FileZilla FTP software. You can upload and concurrently view the contents of the source directory on your machine (in the left pane) and the target directory on CRAN (in the right pane) and view a log of what is happening in the top pane and a progress indicator in the bottom pane. It also has a site manager to store the sites you upload to so you don't need to keep typing in their URL each time you do an upload.