I have an enormous PowerPivot table (839,726 rows), and it is simply too big to copy-paste into a regular spread sheet. I have tried copying it and then reading it directly into R using the line data = read.table("clipboard", header = T), but neither of these approaches work. I am wondering if there is some add-on or method I can use to export my PowerPivot table as a CSV or .xlsx? Thanks very much
Select all the PowerPivot table
Copy the data
Past the data in a text file (for example PPtoR.txt)
Read the text file in R using tab delimiter: read.table("PPtoR.txt", sep="\t"...)
To get a PowerPivot table into Excel:
Create a pivot table based on your PowerPivot data.
Make sure that the pivot table you created has something in values area, but nothing in filters-, columns- or rows areas.
Go to Data > Connections.
Select your Data model and click Properties.
In Usage tab, OLAP Drill Through set the Maximum number of records to retrieve as high as you need (maximum is 9999999 records).
Double-click the measures area in pivot table to drill-through.
another solution is
import the Powerpivot model to PowerBi desktop
export the results from PowerBI desktop using a Powershell script
here is an example
https://github.com/djouallah/PowerBI_Desktop_Export_CSV
A pure Excel / VBA solution is below. This is adapted from the code here to use FileSystemObject and write 1k rows at a time to the file. You'll need to add Microsoft ActiveX Data Objects Library and Microsoft Scripting Runtime as references.
Option Explicit
Public FSO As New FileSystemObject
Public Sub ExportToCsv()
Dim wbTarget As Workbook
Dim ws As Worksheet
Dim rs As Object
Dim sQuery As String
'Suppress alerts and screen updates
With Application
.ScreenUpdating = False
.DisplayAlerts = False
End With
'Bind to active workbook
Set wbTarget = ActiveWorkbook
Err.Clear
On Error GoTo ErrHandler
'Make sure the model is loaded
wbTarget.Model.Initialize
'Send query to the model
sQuery = "EVALUATE <Query>"
Set rs = CreateObject("ADODB.Recordset")
rs.Open sQuery, wbTarget.Model.DataModelConnection.ModelConnection.ADOConnection
Dim CSVData As String
Call WriteRecordsetToCSV(rs, "<ExportPath>", True)
rs.Close
Set rs = Nothing
ExitPoint:
With Application
.ScreenUpdating = True
.DisplayAlerts = True
End With
Set rs = Nothing
Exit Sub
ErrHandler:
MsgBox "An error occured - " & Err.Description, vbOKOnly
Resume ExitPoint
End Sub
Public Sub WriteRecordsetToCSV(rsData As ADODB.Recordset, _
FileName As String, _
Optional ShowColumnNames As Boolean = True, _
Optional NULLStr As String = "")
'Function returns a string to be saved as .CSV file
'Option: save column titles
Dim TxtStr As TextStream
Dim K As Long, CSVData As String
'Open file
Set TxtStr = FSO.CreateTextFile(FileName, True, True)
If ShowColumnNames Then
For K = 0 To rsData.Fields.Count - 1
CSVData = CSVData & ",""" & rsData.Fields(K).Name & """"
Next K
CSVData = Mid(CSVData, 2) & vbNewLine
TxtStr.Write CSVData
End If
Do While rsData.EOF = False
CSVData = """" & rsData.GetString(adClipString, 1000, """,""", """" & vbNewLine & """", NULLStr)
CSVData = Left(CSVData, Len(CSVData) - Iif(rsData.EOF, 3, 2))
TxtStr.Write CSVData
Loop
TxtStr.Close
End Sub
Here is a lovely low-tech way:
https://www.sqlbi.com/articles/linkback-tables-in-powerpivot-for-excel-2013/
I think the process is a little different in Excel 2016. If you have Excel 2016, you just go to the data tab, go to Get External Data, and then Existing Connections (and look under Tables).
The other important thing is to click on Unlink (under Table Tools - Design - External Table Data). This unlinks it from the source data, so it really is just an export.
You can copy that data into another workbook should you wish to.
Data in Power Pivot is modeled, using DAX Studio to export data to csv or SQL.
after finished, you will see that Each model corresponds to a CSV file or SQL table.
Related
I have a large table within a Microsoft Word document.
The majority of rows, but not all, have a single Microsoft Word file attached.
My job is to go into each row and manually type in the file name where an attachment is provided.
Is there any way to automate this task using an R package? For example, for each row that has a file attachment, automatically pull the filename and record it in the field directly to its left?
This is what the table looks like. The files are in the most right column. The column to its left is where I am going to be typing the filenames.
I've tried importing the docx file using the docxtractr package, but it is not reading in the filenames properly. Instead, it is replacing them with \s.
ievs_raw <- read_docx("ievs-raw.docx")
tbls <- docx_extract_all_tbls(real_world)
view(as.data.frame.list(tbls))
Produces the following output with \s where there should be filenames like CAP_ATT_H.11.114.docx etc.:
I wasn't able to figure this out using an R package, but the kind people at the Microsoft Community Forum helped out by providing a super useful Visual Basic Macro. What's great about this is it can accommodate cases where there is more than 1 attachment in a particular row.
Sub ObjectNames()
Dim ILS As InlineShape
Dim nObj As Long
Dim strName As String
Dim col As Long
Dim row As Long
With ActiveDocument.Tables(1)
col = .Columns.Count
For row = 1 To .Rows.Count
strName = ""
# loop through all shapes in this row's last cell
# (if there are none, the loop does nothing)
For nObj = 1 To .Cell(row, col).Range.InlineShapes.Count
Set ILS = .Cell(row, col).Range.InlineShapes(nObj)
If Not ILS.OLEFormat Is Nothing Then
# build up a string with as many names as
# there are embedded objects, separated by
# paragraph marks (vbCr)
If nObj > 1 Then strName = strName & vbCr
strName = strName & ILS.OLEFormat.IconLabel
End If
Next nObj
If Len(strName) > 0 Then
.Cell(row, col - 1).Range.Text = strName
End If
Next row
End With
End Sub
I have trouble with the following tasks in excel VBA:
At my work, we use a document management platform called TeamShare: [https://www.lector.dk/en/products/]
I want to create a code in VBA, that loops over a range of links to this document management platform in my workbook, ie. loops over other workbooks, opens them and then copies a specified sheet to my current workbook.
I have tried putting together bits of codes from other sites, and the code works just fine when i run it in break mode. However, when I run the code all at once, the Excel program reopens, such that the current workbook cannot "communicate" with the opened workbook and I end up in an infinity loop (so no direct error message).
This is the code that only works in break mode:
Dim wbCopyTo As Workbook Dim wsCopyTo As Worksheet Dim i As Long Dim Count As Long Dim WBCount As Long Dim LastRow As Long Dim wb As Workbook Dim ws As Worksheet Dim URL As String Dim IE As Object Dim doc As Object Dim objElement As Object Dim objCollection As Object
Set wbCopyTo = ActiveWorkbook Set wsCopyTo = ActiveSheet
LastRow = wsCopyTo.Range("B" & Rows.Count).End(xlUp).Row
For i = 2 To LastRow
Set IE = CreateObject("InternetExplorer.Application")
IE.Visible = True
The purpose of this piece of code is to get the DocID
A = InStr(wsCopyTo.Range("B" & i), "documentid=") + Len("documentid=")
B = InStrRev(wsCopyTo.Range("B" & i), "&")
DocID = Mid(wsCopyTo.Range("B" & i), A, B - A)
'Get URL
URL = wsCopyTo.Range("B" & i)
'Count number of open workbooks
WBCount = Workbooks.Count
With IE
New is the comman that opens excel sheet. This works as planned in breakmode, however the excel program reopens when i run the code all at once. I have tried other commandos here: "Workbooks.Open", I couldn't get this one to open the file and "Application.FollowHyperlink" only worked in break mode too, however, much much slower
.Navigate URL
'This was my solution to how to stop the rest of the code from executing until the new workbook has loaded.
Do Until Workbooks.Count = WBCount + 1: Loop
End With
'Unload IE
Set IE = Nothing
Set objElement = Nothing
Set objCollection = Nothing
'So in order to activate the workbook from the URL, I am looping over all my open workbooks and matching them on their unique Document ID. I found that the workbook from the URL wasn´t the "active workbook" per default.
For Each book In Workbooks If Mid(book.Name, 12, Len(DocID)) = DocID Then
book.Activate
Set wb = ActiveWorkbook
Set ws = ActiveSheet
End If
Next book
Here i copy the desired sheet to my initial workbook
wb.Worksheets("SpecificSheetIWantToCopy").Copy After:=wbCopyTo.Worksheets("Sheet1") wbCopyTo.Sheets(ActiveSheet.Name).Name = DocID
Next i
End Sub
I am using excel 2010.
I hope you can help me resolve this problem. Please ask if you need any more information, that i haven´t provided.
Thanks in advance.
Background
Suppose I have a shiny app where the user can upload an Excel file. The users will have access to a certain Excel template and I want to make sure that only copies of this template are uploaded.
My current approach
My current approach is now as follows:
Check if sheet name xyz is present -> if not throw an error
Read data from sheet xyz, compare column names with requirements -> if missing columns throw an error
Repeat for all necessary sheets
Problem with the current approach
This requires a lot of hard coding required sheet names and required column names and becomes tedious.
Question
So my question: how can I assure that the user provides a valid file? What strategies do you usually use to make sure that the uploaded file can be properly processed by your apps?
Pseudo Code
library(shiny)
library(tidyverse)
ui <- fluidPage(fileInput("file", "Upload Excel"))
server <- function(input, output, session) {
observe({
req(input$file)
sheet1 <- tryCatch(read_xlsx(input$file$datapath, sheet = "xyz"),
error = function(e) {
## do some sort of error handling, e.g. write to a reactiveValue list
})
if (!all(.REQUIRED_FIELDS_FOR_XYZ %in% names(sheet1))) {
## signal error
}
})
}
If you are already using Excel, why not use a Macro to do the work for you. Consider listing file paths, checking format types, cell addresses, cell values, etc. The Macro below will do most of the heavy lifting for you.
Sub GetFolder_Data_Collection()
Dim colFiles As Collection, c As Range
Dim strPath As String, f, sht As Worksheet
Dim wbSrc As Workbook, wsSrc As Worksheet
Dim rw As Range
Dim sh As Worksheet, flg As Boolean
Set sht = ActiveSheet
strPath = ThisWorkbook.Path
Set colFiles = GetFileMatches(strPath, "*.xlsx", True)
With sht
.Range("A:I").ClearContents
.Range("A1").Resize(1, 5).Value = Array("Name", "Path", "Cell", "Value", "Numberformat")
Set rw = .Rows(2)
End With
For Each f In colFiles
Set wbSrc = Workbooks.Open(f)
Set wsSrc = wbSrc.Sheets(1)
For Each c In wsSrc.Range(wsSrc.Range("A1"), _
wsSrc.Cells(1, Columns.Count).End(xlToLeft)).Cells
rw.Cells(2).Value = wbSrc.Path
sht.Hyperlinks.Add Anchor:=rw.Cells(1), Address:=wbSrc.Path, TextToDisplay:=wbSrc.Name
rw.Cells(3).Value = c.Address(False, False)
rw.Cells(4).Value = c.Value
rw.Cells(5).Value = c.NumberFormat
i = 6
For Each sh In Worksheets
If sh.Name Like "Sheet1*" Or sh.Name Like "*Sheet2*" Then rw.Cells(i).Value = sh.Name & " Exists"
i = i + 1
Next
Set rw = rw.Offset(1, 0)
Next c
wbSrc.Close False
Next f
End Sub
'Return a collection of file objects given a starting folder and a file pattern
' e.g. "*.txt"
'Pass False for last parameter if don't want to check subfolders
Function GetFileMatches(startFolder As String, filePattern As String, _
Optional subFolders As Boolean = True) As Collection
Dim fso, fldr, f, subFldr
Dim colFiles As New Collection
Dim colSub As New Collection
Set fso = CreateObject("scripting.filesystemobject")
colSub.Add startFolder
Do While colSub.Count > 0
Set fldr = fso.GetFolder(colSub(1))
colSub.Remove 1
For Each f In fldr.Files
If UCase(f.Name) Like UCase(filePattern) Then colFiles.Add f
Next f
If subFolders Then
For Each subFldr In fldr.subFolders
colSub.Add subFldr.Path
Next subFldr
End If
Loop
Set GetFileMatches = colFiles
End Function
PUT THIS CODE IN AN XLSB OR XLSM EXCEL FILE IN THE SAME FOLDER AS YOUR EXCEL FILES.
It's probably just easier to do this kind of thing with Excel, and I'm a huge proponent of using the right tool for the job.
I have a bit of a problem and could really use some help. My organization recently migrated from Office 2007 to Office 2010. I had a database that I developed using Access 2007 (using the .accdb database file type). Throughout the migration process, I was still making updates to my database. All the updates were made via an Office 2007 machine and everything worked on the 2010 systems that I deployed it to, as well as the 2007 boxes. The problem now is that since all the computers are officially on 2010, I cannot seem to create an Accde file from Access 2010. The error I receive is: " The command or action 'MakeMDEFile' isn't available now." * You may be in a read-only database or an unconverted database from an earlier version..." The code is compiled with no errors and my references are good. I have tried to re-compile the code, re-name the wizards in the "C:\Program Files\Microsoft Office\Office14\ACCWIZ" folder and let them re-install, and import all my objects into a new database based on this article: http://msdn.microsoft.com/library/office/dn602608%28v=office.14%29.aspx; all to no avail.I do not have any web content or anything Access 2010 specific, as I only made adjustments to the things I created in 2007. I did read that Access must be compiled on the same version that it was created on, but I thought since both 2007 and 2010 use the .accdb file format it would be compatible? Any advice on this? Thank you.
Thank you for your quick answer! That worked great. I was able to successfully export my DB using the code and I also imported all the objects with the exception of the queries. (I was able to create the accde.) Because I have so many objects, I used a script to import everything. The problem I am experiencing now is with my SQL queries. The export script named the text files a little different for the SQL queries and I don't know how to handle them. Below is my code that worked for the rest of the objects:
Public Sub batchImport_queries()
On Error GoTo batchImport_Err
Dim objFS As Object, objFolder As Object
Dim objFiles As Object, objF1 As Object
Dim strFolderPath As String
strFolderPath = "C:\Users\Me\Desktop\dbexport\queries\"
Set objFS = CreateObject("Scripting.FileSystemObject")
Set objFolder = objFS.GetFolder(strFolderPath)
Set objFiles = objFolder.files
For Each objF1 In objFiles
objF1.Name = Right(objF1.Name, Len(objF1.Name) - 6)'strips "Query_"
objF1.Name = Left(objF1.Name, Len(objF1.Name) - 3) 'strips ".txt"
Application.Application.LoadFromText acQuery, objF1.Name, strFolderPath & objF1.Name
Next
Set objF1 = Nothing
Set objFiles = Nothing
Set objFolder = Nothing
Set objFS = Nothing
batchImport_Exit:
Exit Sub
batchImport_Err:
MsgBox Err.Number & " " & Err.Description
Resume batchImport_Exit
End Sub
That worked for queries like: "Query_qryAvailable.txt" but the SQL ones look like this: "Query_~sq_cCIPSSubform~sq_RosterSubform.txt". It seems to be encapsulating "~sq_c" around the first part of the query name and then the form/subform/or control that is associated with it at the last part of the filename...or I could be completely off. I can't figure out the pattern. Some of them have "~sq_f" instead, only at the leading part.(I'm guessing those are for forms?) Anyway, is there a better way to format the file name (if that's what has to be done) to remove those to my original query names and import correctly? Please let me know if that doesn't make sense. Thank you for your time.
It's possible to export an Access database to text files, see here: http://www.access-programmers.co.uk/forums/showthread.php?t=99179.
Option Compare Database
Option Explicit
Public Sub ExportDatabaseObjects()
On Error GoTo Err_ExportDatabaseObjects
Dim db As Database
'Dim db As DAO.Database
Dim td As TableDef
Dim d As Document
Dim c As Container
Dim i As Integer
Dim sExportLocation As String
Set db = CurrentDb()
sExportLocation = "C:\Temp\" 'Do not forget the closing back slash! ie: C:\Temp\
For Each td In db.TableDefs 'Tables
If Left(td.Name, 4) <> "MSys" Then
DoCmd.TransferText acExportDelim, , td.Name, sExportLocation & "Table_" & td.Name & ".txt", True
End If
Next td
Set c = db.Containers("Forms")
For Each d In c.Documents
Application.SaveAsText acForm, d.Name, sExportLocation & "Form_" & d.Name & ".txt"
Next d
Set c = db.Containers("Reports")
For Each d In c.Documents
Application.SaveAsText acReport, d.Name, sExportLocation & "Report_" & d.Name & ".txt"
Next d
Set c = db.Containers("Scripts")
For Each d In c.Documents
Application.SaveAsText acMacro, d.Name, sExportLocation & "Macro_" & d.Name & ".txt"
Next d
Set c = db.Containers("Modules")
For Each d In c.Documents
Application.SaveAsText acModule, d.Name, sExportLocation & "Module_" & d.Name & ".txt"
Next d
For i = 0 To db.QueryDefs.Count - 1
Application.SaveAsText acQuery, db.QueryDefs(i).Name, sExportLocation & "Query_" & db.QueryDefs(i).Name & ".txt"
Next i
Set db = Nothing
Set c = Nothing
MsgBox "All database objects have been exported as a text file to " & sExportLocation, vbInformation
Exit_ExportDatabaseObjects:
Exit Sub
Err_ExportDatabaseObjects:
MsgBox Err.Number & " - " & Err.Description
Resume Exit_ExportDatabaseObjects
End Sub
If the principle still holds, you should be able to import the resulting objects to a fresh database. If there are any problems with permissions, that should become evident in the text files, but normally this strips all permissions.
I'm having a rather weird problem trying to upload an Excel file and storing it in a SQL server table in an ASP.net App.
The file is not too big: about 2.5 or 3 Mb.
The problem is that the upload gets "interrupted" after loading some rows, appearently without causing any specific error, since the load process finishes by showing a success message that I'm sending to the client:
"The process finished successfuly: XXX rows were uploaded from the
file".
The problem is that the XXX rows are not all the rows from the file. Depending on how much information there is in each column of the Excel file, the process is only uploading, for instance, 15500 rows from a total of 25000 that the file has. (If there's less information in the Excel file, it can upload, lets say 20000 of the 25000 rows that it may have)... the fact is that the file is not being "completely" uploaded.
I already tried increasing the "httprequest-maxRequestLength" value in the web.config, even though the file is not bigger than the default 4Mb file upload that ASP.net has.
The code (vb.net) that I'm using upload and read the file is, basically, this:
Dim connection As DbConnection = Nothing
Dim Command As DbCommand = Nothing
Dim ConexionStringExcel As String
Dim dr As DbDataReader
Dim mensajeError as String = ""
Dim ncAdic as Integer = 0
Dim nReg as Integer = 0
'String connection for Excel 2007: for now, I'm not allowing other Excel versions
ConexionStringExcel = "Provider=Microsoft.ACE.OLEDB.12.0;" & _
"Data Source = " & sNombreArch & ";" & _
"Extended Properties=Excel 12.0 Xml;"
'sNombreArch is the full name of the uploaded file
connection = New OleDb.OleDbConnection(ConexionStringExcel)
Command = connection.CreateCommand()
Command.CommandText = "SELECT * FROM [" + nomHojaArch + "$]"
'---
'lblMensaje is a Label object in the aspx page:
'---
lblMensaje.Visible = False
Try
'Open the Excel file
connection.Open()
'Read file content
dr = Command.ExecuteReader()
While dr.Read
Try
'Two first columns of the file are mandatory in my case...
If Not IsDBNull(dr(0)) And Not IsDBNull(dr(1)) Then
'---
'dsTempCargue is a SqlDataSource object in the aspx page
'---
dsTempCargue.InsertParameters.Item("idReg").DefaultValue = dr(0)
dsTempCargue.InsertParameters.Item("nombre").DefaultValue = dr(1)
For ncAdic = 2 To 10
If Not IsDBNull(dr(ncAdic)) Then
dsTempCargue.InsertParameters.Item(ncAdic).DefaultValue = dr(ncAdic)
Else
dsTempCargue.InsertParameters.Item(ncAdic).DefaultValue = DBNull.ToString
End if
Next
dsTempCargue.Insert()
nReg = nReg + 1
Else
mensajeError = "Column A and B of the File cannot be empty"
Exit While
End If
Catch exRead As Exception
mensajeError = "Error reading the file or saving its content: " & exRead.Message
Exit While
End Try
End While
'If there was no error, show success message
If String.IsNullOrEmpty(mensajeError) Then
mensajeError = "The process finished successfuly. " & nReg.ToString() & " rows were uploaded from the file"
End IF
Catch ex As Exception
mensajeError = "Error uploading the file: " & ex.Message
End Try
lblMensaje.Text = mensajeError
lblMensaje.Visible = True
Why do you think this uploading process is failing to read the entire file???... Any advice will be appreciated.
Thanks a lot,
Diego
For the tables that you are storing the rows in, what datatype are the various columns? Perhaps the content of one of the cells is incompatible with the column datatype. Do you have any way of checking?
Even though I'm really not sure about what could have been the cause of the error, I finally managed to get it working by changing my code to use a Third-party library that I found and that had some good comments, which is available here:
http://exceldatareader.codeplex.com/
I'm not really sure about what the original problem was and why using this library solved the issue, but It worked. (Perhaps when I have some time I'll try to find more info about this problem I had)
Thanks Patrick for your comments and all for the interest