I've a clustered partitioned table exported from GA 360. Attached is the image. I would like to create exact replica of the same. Using Web UI it's not possible. I created backup table using bq command line tool, still no luck.
Also, whenever we check preview it has a day filter. It looks like this:
Whenever data is appended to the backup table, I don't find this filter there even though this option is set to true while creating a table.
If you can give more context about handling this kind of table it would be beneficial.
Those are indeed sharded tables. As explained by #N. L they follow a time-based naming approach: [PREFIX]_YYYYMMDD. They then get grouped together. The explained procedure to backup them seems correct. Anyhow, I would recommend to use partitioned tables as it will be easier to backup them and they perform better in general.
This is not a cluster / partitioned table. This one is a sharded non-partitioned table having one common prefix. Once you start creating multiple tables with same prefix we can see them under the same prefix.
Ex:
ga_session_20190101
ga_session_20190102
both these tables will be grouped together.
To take backup of these tables you need to create a script to copy source to destination table with same name and execute that script using bq command line tool under the same project.
Related
I have seen several members on this forum warn about creating tables in a database while the app is running. Instead it is encouraged to create the tables while programming, and only fill the tables with data during runtime.
E.g while creating a note-app it would be convenient to let the user specify a name for a single note, and let this note be created as a table in a database. This by creating the table at the time the user creates the note, and letting the name of the note be the name of the table. Why would this be a bad practise? Have I misunderstood?
It would be highly inconvenient for both you and the user of such an app to create a table for every note the user might want to add. It's just not the way it works. A table should contain rows of information of the same type, such as a note for example, and each note should be added as a row/record in the said table. The table should be called notes for example, and if you want a name for each note, it can be a column in the notes table called name.
An analogy would be, if you are taking notes manually (without an electronic device that is), would you have one notebook with you and just add notes on different pages as you need to, or would you carry around a bag full of notebooks so that whenever you want to add a new note, you would add each note in a separate notebook?
The notebooks being equivalent to database tables in this analogy, and the pages of the said notebook being equivalent to rows in a database table.
I can't think of a reason for creating tables during runtime really. The database structure should be "set in stone" so to speak, and during runtime you should only manipulate the data in the database, which is adding, deleting, or updating rows/records in already existing tables. Creating tables during runtime is a big no-no.
I probably don't understand your question correctly, but it seems to me, that what you really want to do is to create a new row in a table I the database?
Working on a SQLite database, seems DbVisualizer Pro does a lot of work very well, except one,
Changing table schema.
I often need to change column name, data type, etc, but don't want to do it through raw SQL statement. My workaround is opening Firefox's SQLite Manager to just change the schema.
Is it possible to use DbVisualizer to change the schema? many thanks!
Edit:
Alter table action mentioned below by roger, seems to be the right way to go. But somehow I can only add column, the existing column appears to be read only.
Mine is DBVisualizer Pro Evaluation. Is non-Evaluation different?
Edit2:
Using SQLite Manager is sometimes dangerous, as warned below. just learned, renaming a column may cause the foreign key loss. but workaround is here
In DbVisualizer Pro there is the Alter Table action (and Create Table for creating new tables). Select the actual table you want to change in the Databases tab, right-click and chose Alter Table. In order for this to work you need DbVisualizer Pro and the Database Type for your connection must be set to either Auto Detect (recommended) or SQLite.
I have a DynamoDB table called Transactions in which I am recording an audit trail of activity in my app. After midnight I would like to rename the table to something like Transactions.2015-10-01 and create a new table called Transactions, to which I would record the next 24 hours of activity in my app. At the end of that period I would rename that to Transactions.2015-10-02 etc.
In other words, I'm trying to implement a rotating log of activity (though linear rather than circular).
I'm not sure there's a way to rename a table in DynamoDB. Is there a way? If not, is there another approach? For example, if there was a table pointer that was called Transactions that would point to Transactions.2015-10-01 and then at midnight that pointer would switch to to Transactions.2015-10-02. My app audit logic would simply perform insertions into Transactions which delegates to the right table.
Amazon DynamoDB does not have a command to rename tables. The only table-related commands are CreateTable, DeleteTable, DescribeTable, ListTables and UpdateTable (which does not permit a rename).
Nor is there the ability to create aliases to tables.
The best approach would be for your application to create the desired tables and then direct transactions to the appropriate table.
Not exactly a work around, but you can create a backup, restore the backup to a new table with a new name, then optionally remove the original table.
This is not a "rename", and it could break a lot of things. Just a hint. Don't go ahead and do it on your production server!
I am designing a standard ASP.Net site with a SQL database. I have a database schema and During the tests I am changing data types amongst other tasks and the data contained inside really is not that important.
I keep getting errors as the old data does not match the new rules. This is not important and I am happy to clear everything but currently, I have to export/publish the database to a .sql file then import it from scratch - which is time consuming.
Is there a quick button / feature that I have missed that allows you to reset autonumbers / IDs to 1 and delete all content, or just speed up what I currently do?
There are a few options you could take, the "fastest" really depends on your database.
To firstly answer your questions on seeding, etc - TRUNCATE TABLE will delete all information in a table (very fast, as it is not logged) and will reset your identity column.
eg:
TRUNCATE TABLE dbo.table
http://msdn.microsoft.com/en-us/library/aa260621(SQL.80).aspx
The significant restriction here is that you cannot use it on a table that is referenced by another table. In this case you can use a standard delete and then use DBCC CHECKIDENT
eg:
DELETE FROM dbo.table
GO
DBCC CHECKIDENT(dbo.table, reseed, 0)
http://msdn.microsoft.com/en-us/library/ms176057.aspx
Remember with delete to make sure you delete information in the correct order (i.e. taking into account foreign keys).
Another approach I often use is simply writing a complete tear-down / rebuild script when I want to reset the database. The basic premise is to tear down, or drop all database objects at the beginning of the script and then recreate them. This is not necessarily a solution for all scenarios, but for basic tasks works well for me. To avoid errors I would usually add my drop statements in IF statements, eg:
IF EXISTS
(
SELECT *
FROM information_schema.tables
WHERE table_name = 'table' AND table_schema = 'dbo'
)
BEGIN
DROP TABLE dbo.table
END
Why don't you write some T-SQL code to delete (or truncate, even quicker) all your tables? Be careful to take into consideration your integrity rules while clearing the tables: allways clean the tables containing the foreign key before cleaning the one containing the primary key.
If you just need to clear out data then just write a script to truncate all the data in each table. The truncate command also resets any IDENTITY fields as well.
TRUNCATE TABLE myTable
For each table you have. Then just run that script each time.
Here'a a quick way to delete all of the data in a table:
TRUNCATE TABLE YourTableName
You could write a script that would truncate all of your tables.
The alternative is to just DROP the table and re-create it.
If you really want to drop all data, then you could detach the database and create a brand new one; it's a bit extreme, but possibly faster than dropping everything first.
As others have suggested I find it preferable to maintain a script that builds the database from scratch and can tear down the database prior to rebuilding it. Develop this script just as you'd develop the rest of the application. I find it easier to understand the database through a script than by building it through a GUI, especially where there are complex relationships, triggers and so on.
It's also useful if you have other developers, and perhaps quicker and less prone to errors than copying your working database and handing it to another developer.
On release you can freeze that script and then create delta scripts for the next release which has just the changes from the initial schema to the new. This could also tear down the new objects created in the delta before recreating them so it can be easily re-run without having to wipe the entire database.
if you use Visual Studio 2010 then
open the App_Data folder of the solution and double click on the MDF File.
right click on your table , in the menu select "Show Table Data".
select all rows and delete all them.
I am re-designing an application for a ASP.NET CMS that I really don't like. I have made som improvements in performance only to discover that not only does this CMS use MS SQL but some users "simply" use MS Access database.
The problem is that I have some tables which I inner join, that with the MS Access version are in two different files. I am not allowed to simply move the tables to the other mdb file.
I am now trying to figure out a good way to "inner join" across multiple access db files?
It would really be a pity if I have fetch all the data and the do it programmatically!
Thanks
You don't need linked tables at all. There are two approaches to using data from different MDBs that can be used without a linked table. The first is to use "IN 'c:\MyDBs\Access.mdb'" in the FROM clause of your SQL. One of your saved queries would be like:
SELECT MyTable.*
FROM MyTable IN 'c:\MyDBs\Access.mdb'
and the other saved query would be:
SELECT OtherTable.*
FROM OtherTable IN 'c:\MyDBs\Other.mdb'
You could then save those queries, and then use the saved queries to join the two tables.
Alternatively, you can manage it all in a single SQL statement by specifying the path to the source MDB for each table in the FROM clause thus:
SELECT MyTable.ID, OtherTable.OtherField
FROM [c:\MyDBs\Access.mdb].MyTable
INNER JOIN [c:\MyDBs\Other.mdb].OtherTable ON MyTable.ID = OtherTable.ID
Keep one thing in mind, though:
The Jet query optimizer won't necessarily be able to use the indexes from these tables for the join (whether it will use them for criteria on individual fields is another question), so this could be extremely slow (in my tests, it's not, but I'm not using big datasets to test). But that performance issue applies to linked tables, too.
If you have access to the MDBs, and are able to change them, you might consider using Linked Tables. Access provides the ability to link to external data (in other MDBs, in Excel files, even in SQL Server or Oracle), and then you can perform your joins against the links.
I'd strongly encourage performance testing such an option. If it's feasible to migrate users of the Access databases to another system (even SQL Express), that would also be preferable -- last I checked, there are no 64-bit JET drivers for ODBC anymore, so if the app is ever hosted in a 64-bit environment, these users will be hosed.
Inside one access DB you can create "linked tables" that point to the other DB. You should (I think) be able to query the tables as if they both existed in the same DB.
It does mean you have to change one of the DBs to create the virtual table, but at least you're not actually moving the data, just making a pointer to it
Within Access, you can add remote tables through the "Linked Table Manager". You could add the links to one Access file or the other, or you could create a new Access file that references the tables in both files. After this is done, the inner-join queries are no different than doing them in a single database.