I have a table with some data already. Then I created a few indexes over time. The oldest index has all the items, but two other indexes I created recently were empty. Today I deleted those two indexes and created again. Now they have only 19 items out of 1564 in the table.
I have the same structure but in different region for testing and it all works just fine.
Can somebody explain what's going on, please?
Edit: Here are screenshots of the table and indexes. Number has increased due to new items been added to the table.
Related
I have an AS tabular model that contains a fact table with 20 mil rows. I have partitioned this so only the new rows get added to each day... however occasionally, a historical row (from years ago) will be modified. I can identify this modified row in SQL (using the last modified timestamp) however would it be possible for me to refresh the row in SSAS to reflect this change without having to refresh my entire data model? How would I achieve this?
First, 20 million rows is not a lot. I’m expecting that will only take 5-10 minutes to process unless your SQL queries are very inefficient or very wide. So why bother to optimize something which may be fast enough already?
If you do need to optimize it, you will first want to partition the large fact table by some date element. Since you only have 20 million rows I would suggest partitioning by year. Optimal compression will be achieved with around 8 million rows per partition. Over-partitioning (such as creating thousands of daily partitions) is counter-productive.
When a new row is added you could perform a ProcessAdd to insert just the new records to the partitions in question. However I would recommend just doing a ProcessFull on any year partitions which have any inserts, updates or deletes in SQL.
SSAS doesn’t support updating a specific row. Thus you have to follow the ProcessFull advice above.
There are several code examples including this one which may help you.
Again this may be overkill if you only have 20 million rows.
SHORT VERSION: If all else fails, add a value (even zero) to an additional Number column in the SQLite table to stop the #DELETED demon.
LONGER VERSION: There have been many posts about this frustrating and inconsistent problem over the years. Lord knows, I have read each one at least half dozen times and tried every remedy proposed - and usually one of the incantations will finally solve the problem. Yet I found myself recently in a new quandary and spent the last two days racking my brain to osmose why none of the tricks worked on it.
It was classic: Access 2019 front end linked to a SQLite back end table (via the Devart SQLite ODBC driver, which I believe is inconsequential). The table had the recommended col_ID of Number format, Auto-incrementing as the Primary Key. It had col_DateUpdated, Text format with default value of current_timestamp. There was col_OperatorID which was Text format and the only column in a Unique Index. Finally, there were two other non-indexed Number format columns.
The table worked fine in SQLite. I could Add, Delete, Update no problem. In Access, when I opened the linked table and added a record it did not immediately show the auto incrementing value in col_ID, nor the date/time stamp. When I clicked off the row, it immediately filled all the columns of the new row with #DELETED. If I closed the table and reopened it, the new row would display just fine.
The SQLite Pragmas were all set to Default including Auto Index, Normal Locking Mode and Full Synch. I tried every combination of changing the table structure, column formats, indexes, default values, etc. The problem persisted regardless of whether there was any other data in the table or not.
I've been coding in Access for over 30 years and SQLite for three and have never seen anything like it.
I was stumped until , for the heck of it, I added a value into one of the other Number columns. Amazingly, it worked great!
I can create a new row, put values in col_OperatorID AND the two non-indexed Number columns, click off the row and it takes it fine. It updates the autonumber primary key col_ID and col_DateUpdated with the current date/time just fine with no #DELETED nonsense.
It beats me why it works, maybe Access finally can accept it as a really, really unique record (even though the additiaonal data is not in any index) or maybe putting the numeric value in the other, seemingly unimportant, columns forces an update across the link, I don't know. But I thought I would pass this along because I KNOW probably forevermore, unless Microsoft or the SQLite folks come up with a cure for this, there will be people that will need this additional gimmick to get out of #DELETED hell.
Good luck and Happy Trails.
I'm a bit inexperienced but have a managed to learn how to use my database (access2010) but now I need to remove old files. In the database I have a primary table with multiple tables which stores additional information such as my notes.
I can't seem to figure out how to remove old files based on an input date.
I want to remove all files and the data stored in the dependent tables completely from year 2011 and back after backing up the database.
I've tried a delete query, and I've tried to simply copy and past inside the tables. I know there has to be a way to do this without deleting individual files.
When I run a delete query I get invalid key errors and when I delete files from my primary table, I get errors indicating there are associated data stored in the other tables.
Since I can't seem to delete all data across all tables for a certain date range, can anyone point out what I might be doing wrong?
You will need to JOIN the additional tables to your "primary" table, include a WHERE clause to only delete those matching your date range.
For an example, see this answer https://dba.stackexchange.com/a/102738
Before I explain the problem, let me set the context. I have a stored procedure which returns me paged results. So if I mentioned the page number as 1, I get the first 10 records and if the page number is 2 I get the next 10 set of records and so on.
Now in my Asp.Net page I need to show 10 items at a time and I want the pagination to be in numbered style. The same like Google pagination.
The problem now is that, since I get only 10 records at a time from the DB, the numbers don't show up. I've come across a solution at: How to use DataPager with Database Paged, however since my total number of records can run into 100's or 1000's, running the loop for so long is again a performance issue.
It would be great if you can let me know how this issue can be fixed.
Thanks in advance.
Tweak your stored procedure to add an output parameter that returns the total number of records that will be returned without pagination. You can then use that number to create your pager controls.
I am encountering a performance problem in the development of current UI. The problem, I suppose, is however general.
I have a page with a simple asp.net grid. Grid will display data from a table based on certain search criteria. Moreover, grid has fixed page size (say 10). There is pager at the bottom which can be used to navigate b/w pages. On the back end, whenever search button is pressed a stored procedure is called which returns the desired data.
Stored procedure has parameters like currentpageIndex, pagesize, other search criteria, etc. Here is a pseudo code for sp:
-- SP begins
-- calculate the page index range to return required using current page index and page size
-- query the table in a CTE and do all filtering. Also calculate row numbers so that
-- correct record range can be returned.
-- use the cte to return the correct record based on the row number calculated in CTE
-- SP ends
I have following problems/queries in this approach
When Db table size is large (say 10 million records), performance degrades and this approach becomes impractical.
Is using table variable or a temporary table more useful?
Is there any other efficient way to get paged data from database?
Hi Dan, the article provided a new insight for calculation of total rows. Really helpful. Thanks.
But still is there better way than using CTE when data is large?
Update: I have found few other performant approaches for efficiently getting paged records.
There's a good article on SqlServerCentral called SQL Server 2005 Paging – The Holy Grail that shows a few techniques for server-side paging. You will need to register to view it, though.
I know for really large result sets then software like Google will simply estimate how many rows will be returned, bypassing the need to get a count of all the rows returned.
Sorry, if I can't give more help.