The state of Maine publishes its public documents as Word .docx files. These documents appear hierarchically structured, but the structure is just visual, not encoded in the document as styles. I'd like to programmatically convert/process one of these documents so that the structured appearance is encoded in the document. The resulting file can be in a file format other than .docx, in fact I'd prefer it since I'd ultimately like to publish it on the internet.
I don't really want to code a parser/converter from scratch, so I wonder if someone has solved this problem using open source tools?
Related
Correct me if I'm wrong: The examples online are generally for a static data set. The executed R code reads data into an R data frame and creates the html widgets to plot it. Upon rendering the .Rmd or .qmd, that data set is baked into the html document. If you wanted the document to show some new data, you'd have to run the R code again.
Is there a way to create a rendered .html with html widgets that's linked to a live .csv or SQL database? Perhaps there are some R tools that upon rendering the document, create the html/js instructions to read a .csv or .json file?
Looking to have an .html file that can be shared or hosted that doesn't require an R interpreter.
I have learned that .docx files are basically binary files. But I'm unaware of the structure that lies beneath.
What is the essential structure of a .docx file? Like, how long is the header? From what point does the actual document content start? Does it have any signature at the end?
Basically, what's the anatomy of a .docx file?
Docx is basically a zip archive with a lot of xml files in it. It is an open format and the documentation is available online. The wikipedia article has a general description and the links you will need.
I am going to answer this question: "What's the Anatomy of a DocX File?"
Official Answer
Please see the official OOXML article, "Anatomy of OOXML," for an example DocX directory structure :
http://officeopenxml.com/anatomyofOOXML.php
For an example DocX XML document :
http://officeopenxml.com/WPsampleDoc.php
What I Personally Suggest
HOWEVER, after following these meticulously, and guessing where the details got foggy, I was unable to make the .docx file.
I chose this short cut : Make a Docx file in Libre Office (supports .docx extensions), make a generic template in the format of the docx files you expect to be generating, save the file as .docx, copy and save as .zip.
Open this .zip directory, and what you'll see I found to be much better at explaining the spec than the above, official links.
Example
For example, if you're making articles in .docx, you'd have [[Title]] at the top in title-casing/formatting, By: [[Author]], for author, etc., etc.. Then with your code, use that template, and just swap out the [[field]] for whatever $data you're ready to put into it.
Can you use PurePDF to view files or is the api only for writing them?
Based on the PurePDF Project Page, reading and extracting information from PDFs is supported:
read existing pdf documents (extract strings, streams, images and all the informations from them). See HelloWorldReader.as for an example
However, if you're looking to view / rasterize a PDF, that's a much more complicated task and doesn't look like it's supported as part of PurePDF.
I suggest converting the PDF into a swf file. There are a number of projects out there (including free / open source) that convert pages into SWF files, including being able to still extract the text. :D
It looks like you can either navigate to the url of the PDF (maybe in an HTML component?) , OR a richer solution might be to use the open source flex paper : http://flexpaper.devaldi.com/
The last time I produced a catalog I used a software called EasyCatalog that worked with Adobe InDesign to merge data from a spreadsheet with graphics. I wouldn’t say it was completely successful. I know of one other catalog building software called Catalog Builder by Computer Pundits. I'm just looking for any suggestions from someone who might have gone through this process on what software I should use.
InDesign can create a really beautiful output from XML. Depending on the catalog content's complexity, you can either have a straightforward mapping of the elements in your catalog to the paragraph and character styles of the IDD file, or you may need to preprocess the XML with XSLT.
For example, if your data source can output the content as XML, but it doesn't map easily to InDesign tables, XSLT can be used to make the XML more "IDD-friendly" before you import it.
IDML is another way to handle XML content; instead of importing the XML content manually or with a script into your catalog template, you generate the IDML directly from your XML. (IDML files are a package of XML files that describe the page/spreads, fonts, swatches, text, images, etc. of the InDesign file.) You're probably going to need XSLT consulting help if this is not a skill you already have.
Take a look at the InDesign documentation for XML for the version you use. IDML is for CS4 or CS5.
Have a look at xCS.press by a company in Belgium. XML markup that is parsed to InDesign. Great for product catalogs.
I wouldn't use Catalog Builder by Computer Pundits again. I've used it in the past (mostly their website builder) and it is completely outdated in my opinion. Their templates are not easily customized and it was pretty slow for me. As for their website builder,(in case you're wondering) it's all tables and very little css IDs or Classes throughout the html.
I haven't used InDesign, but it seems there are lot of scripting features.
Easiest thing that comes to mind is creating an XML Schema with IDML to
get data from Excel into an InDesign document.
XML schema basically is a template for XML documents, and they're called XML Maps in Microsoft Office.
Am not familiar with catalog tools, try superuser.com as well for 3rd party tools & tips.
John. have you come across additional solutions since posting this? I wonder if you have considered CatBase or EM Software solutions.
InDesign works wonderfully with XML and XSLT. You can export the data from Windows Excel only to XML, but only when you create an XML-compatible worksheet. Don't save the file as an XML spreadsheet, that file is useless in InDesign.
What I do is either create a schema file (xsd) for the data that you want to use and import that into Excel on Windows (Mac version doesn't support XML) Once the schema is imported you can create an XML worksheet based on this schema and then copy and paste the data from the non-XML worksheet into the XML sheet. Once the data is in the spreadsheet you can export to an XML file and import it into InDesign.
As mentioned above you can map XML tags to Paragraph and character styles and create dynamic layout directly in InDesign or by using an XSLT to structure the data before you import it.
MS Access allows you to export directly to XML. If you move your data to InDesign you can save the time needed to build the XML spreadsheet. Image references have to be built properly before you export to XML or build an XSLT that will do it on the fly as you import the data in to your layout.
The entire process is described in detail in the book A Designer's Guide to Adobe InDesign and XML.
If the data is in MS Access Woodwing has a product that allows you to interface and import data for a catalog. I have not used it personally but I know people who have. Also, another product called In-Data also interfaces with InDesign, but I have no experience with that either. I usually just use XML and XSLT myself.
I've used EasyCatalog very successfully for a number of years now, even for really large catalogs (35,000+ articles). In the meantime, I offer EC consulting and hands-on user training as well.
I'd need many more details of what went wrong with your specific catalog in order to be able to point your attention to a different solution that may better fit your needs.
I personally would not recommend Jim Maivalds solution because a) Excel and Unicode are not friends b) working with Excel and XML really is a pain c) the process is relatively complicated d) you need a lot of specialized skills regarding XML, XSLT programming and so on e) it's not bi-directional f) when updating you'll do the whole process again.
With EasyCatalog, you just import your data into a panel and place them from there into your document, from manually up to fully automatically. It's really easy, and it's bi-directional - so you can update your document from the database at any time and - if you need to - your database from your document. By the way, you can import your data directly from Excel into an EasyCatalog panel as well.
However, EasyCatalog might not be the best solution if graphics are included in you spreadsheet as well - but who would ever include the real graphics in a spreadsheet instead of the name (and maybe path) to the actual graphic files?
What my users will do is select a PDF document on their machine, upload it to my website, where I will convert into an HTML document for display on the website. The document will be stored in a database after conversion.
What's the best way to convert a PDF to HTML?
I have been handed a requirement where a user would create a "news" story as a pdf and then would upload it to the sever, where it will be converted to HTML and displayed on the website.
Any document creation software that can save documents as PDF can save them as HTML. I'm assuming the issue is that your users will be creating rich documents (lots of embedded images), which results in multiple files, and your requirements stem from a desire to make uploading these documents as simple as possible to the user.
There are numerous conversion packages that can probably do this for you, however when you're talking about rich content, you are talking about text plus images. Those images have to be stored somewhere and served somehow, and whatever conversion method you use will require you to examine all image sources to make sure they point to valid locations on your server.
I would like to suggest an alternate way of doing this that you can take to your team: Implement one of the many blog APIs for publishing content. There are free and commercial software packages that use these APIs to publish content directly to a website, such as Windows Live Writer and Microsoft Word. Your users can simply create their content and upload it directly to your website without having to publish it as PDF first then upload it. So the process becomes much smoother for your users, and you get the posts in a form that doesn't require you spend thousands of dollars on developing or buying conversion code.
The two most common APIs are the MetaWeblog API and the Movable Type API. Both are very simple and easy to implement. I think this way would be a MUCH better alternative than what you're thinking about doing.
I don't think converting a PDF to an HTML string is necessarily the best idea, especially if you want to export it back as PDF. PDF files often contain binary elements such as images, so you may be best to convert it to ASCII via an encoding, such as Base64. That way you will have an ASCII string you can save into a text field in the DB and then convert it back out. Could you expand more on the main requirement?
My recommendation would be to not do it that way IF POSSIBLE (but we all know what managers are like) so...
I would recommend that you stay away from converting the PDF to/from HTML (because unless you can find a commercial solution it will be nigh on impossible) and instead do as has already been mentioned and store it as an encoded Base64 string, or BLOB or some other binary format in the database, and then display it to the user with some sort of PDF view plugin for the browser.
All it took was a simple google search for "PDF to HTML": http://www.gnostice.com/pdf2manyOverview_x.asp. I'm sure there are others.
So while it's 'possible', you may want to explain to your manager that this isn't the best content management solution.
Why not use the iTextSharp to read the PDF content? Then You could save both the binary PDF and the text content to the database. You could then let users search the content and download the PDF.
You should look into DynamicPDF. They have a converter (currently Beta) out for serving exactly this purpose. We have used their products with great success (especially for dumping Reporting Services reports directly to PDF).
Ref: http://www.dynamicpdf.com/