3D File Format For Long Term Storage

When Adobe added 3D to PDF they hoped that CAD users would start using 3D PDF as a data exchange and storage format, instead of IGES and STEP or some CAD vendors proprietary format like Parasolid, JT, etc. To aid the adoption of 3D PDF Adobe decided to make it a truly open standard by making it an ISO standard. In a post titled “PDF To Wear The ISO Hat” that I published here back in December 2007 I wrote:

“Adobe is supposed to have converted the PDF 1.7 Reference into an ISO Draft by removing all Adobe and Acrobat specific information. Once PDF becomes an ISO standard, Adobe will be just one of many implementers of that standard.”

I have been talking to Visual Technology Services, a UK based company which is another implementer of the PDF standard besides Adobe. They have developed a C++ 3D PDF authoring  library called PDF3D SDK aimed at CAD software developers who wish to add 3D PDF publishing capability to their products. They also have an end user product called PDF3D ReportGen and some plug-ins to create 3D PDF files. Today Visual Technology Services issued a press release announcing compatibility of their technology with the latest version of Adobe Reader X including PRC (Product Representation Compact) encoding for 3D models. This proves that PDF is truly an open standard.

I often get asked this question, “What 3D file format should I use to for long term storage of my 3D CAD data?” My answer always is “3D PDF or 3DM“.

3D PDF

Even if Adobe disappears or is bought by a company like Enron or does something stupid (see “The Adobe 3D PDF Train Wreck“), the latter being the one with the highest probability, there will always be a way to get access to your 3D data since PDF is now a truly open ISO standard (ISO 32000-1:2008). The PDF file format specification is out there for everyone to see. Visual Technology Services made use of the ISO PDF spec document to create their own 3D PDF library and end user software.

This is probably one of the reasons huge companies like Boeing have invested heavily into 3D PDF. These companies need to be sure that they will continue to have access to their data 50 years from today. Some people use IGES and STEP to store their data for the long term. I hope you don’t. And if you absolutely had to choose between the two I would recommend STEP over IGES. Over the years CAD vendors have abused the IGES file format by making variants of it to suit their specific needs, thereby defeating the very purpose of a neutral standard. One look at the Rhinoceros IGES export options dialog box gives you an idea of the magnitude of the problem.

There are 60 types of “standard” IGES files and I’m sure more will be added in time. Alibre has created its own flavor of the STEP file format to store certain parameters. I’m not sure whether any other CAD vendors have done something similar. It looks like STEP is going down the path of IGES. While these CAD vendors may have the best of intentions it really does not help creating different flavors of a neutral file format. Having variants of a standard is it itself an oxymoron.

3DM

What Adobe did with PDF is truly commendable. They took their PDF proprietary file format and gave it to ISO to make it an open standard. McNeel did something even more commendable with Rhinoceros by giving away the source code of their 3DM read/write library through the OpenNURBS Initiative. As a programmer if I want to write software to read or write a 3DM file I do not need to read some 3DM file format specification and cook up the source code for it. I simply use the source code  given by McNeel. In fact that is precisely what we do at SYCODE. Many times, when our data exchange products convert a 3D CAD file from one format to another we use 3DM as an intermediate format. For us, 3DM is a truly open standard 3D file format.

There are other open or semi-open formats as well. Autodesk’s DWF is one example. But 3D PDF and 3DM are the ones I recommend the most. The only reason I put 3D PDF over 3DM is because the future of PDF is not guided by a single entity or company. If something is to be made a standard it not only needs to be used by many people. It should also be developed and driven by many people.

Proprietary Standards

The problem with many CAD vendors and their “propreitary standard” formats is that they want to keep total control over them with the result that they becomes a standard for themselves and those in their ecosystem only. Take JT from Siemens PLM for example. First of all you need to pay Siemens PLM an initial membership fee plus an annual maintenance fee to get into their JTOpen program. And to make it worse, these fees are calculated on your annual revenue. There’s more. If you use their software to develop in-house software or software that you share with “corporate” members of the JTOpen program, you are good. Otherwise you need to pay Siemens PLM royalty. I often refer to JT as “Just Talk” because Siemens PLM’s interest in JT is more about earning revenue from it and less about solving the data exchange problem of the CAD software industry.

What’s your 3D file format for long term storage? Do you think about long term storage in the first place?

  • http://twitter.com/CADkid Marijn

    I always come across vrml this post made me think and I looked vrml up and it is iso and open standard it is only superceded by X3D. I guess these aren’t really CAD files but more general 3D files. Don’t know which limitations this brings?

    As a Autocad user we just use dwg format, not even DXF. But I could batch a whole database in 1 day to dxf.

  • http://wertel.eng.pro Scott Wertel

    LOTAR (LOng Term Archiving and Retrieval) was a big topic at GPDIS this year. My impression from that is JT, with its move to an ISO standard, is a reasonable alternative barring the notes you make above about it not being truly open. But the real winner I think is the STEP standard. STEP 203 Edition 2 is a great LOTAR standard. STEP 214 isn’t as well suited, but thankfully STEP 242 will be combining both of those standards into a new MCAD standard.

    STEP 242, I believe, will be the neutral format for LOTAR. The biggest problem is that CAD vendors haven’t even implemented the full STEP 203 Edition 2 standard into their products. If software vendors don’t implement a standard in its entirety, what’s the point of a standard?

  • http://www.deelip.com Deelip Menezes

    AFAIK VRML is a visualization (mesh) format. If you are only interested in visualization data then STL will be your best bet.

  • http://www.deelip.com Deelip Menezes

    Scott: “If software vendors don’t implement a standard in its entirety, what’s the point of a standard?”

    That is the problem Adobne faces with 3D PDF. They thought that all the CAD vendors would license their PDF SDK and that 3D PDF would automatically become the standard.

  • proepro

    We leave it in the original format and try to keep the old software running. We haven’t tried it yet but we are hoping that we can start moving them to virtual machines.
    While this lets you revise old drawings it does not make it easy to reuse old models unless they can be exported to a neutral format (and are worth exporting).

  • http://www.deelip.com Deelip Menezes

    I’m afraid the “intelligence” that you give to your geometry is locked up in the proprietary file format. If you need that you will have to stick to the native file format of the software where you created it.

  • http://wertel.eng.pro Scott Wertel

    That’s not true. STEP 203 and the newer 242 (when released) includes much of the intelligence of the native model. It may not contain features, but that’s not intelligence required for long term archiving and retrieval. Intelligence is the dimensions, tolerances, and other manufacturing data. Five years from now, I don’t care at all about the recipe used to create the geometry.

    And we have to remember, it’s not just about Long Term Archiving. I can put a binary code onto magnetic tape and archive it for as long as the storage medium lasts. The point is also Retrieval. Long term storage of data on a disc is no good if I can’t retrieve it. But I guess it makes a great paper weight.

  • http://www.deelip.com Deelip Menezes

    True. It depends on what kind of intelligence you are looking for. If its just dimensions, tolerances and PMI, most neutral formats can handle that. If you want features then you have a problem.

    The ability to retrieve data from a magnetic tape is no use if there is no current software capable of reading it.

  • Tony

    Why not archive to multiple standards? That should increase your retrieval chances in the future. I’d say use 3D PDF and STEP AP203E2/AP242, plus maybe a more “neutral proprietary” format such as ACIS or Parasolid.

    3D PDF has one big plus: readily available viewers.

    Another point: you should verify how the models are exported. IIRC, 3D PDF can handle both very accurate BREP models as well as “for representation only” mesh models.

  • Anonymous

    Hi Deelip:
    Thank you for your discussion of this subject. Kubotek is passionate about interoperability, and management of change (intended and unintended change) and for this reason Kubotek has been active participant and supporter of the LOTAR International.
    Here are some key points to evaluate as you chose which files to store your data in for Long Term Archival and Retrieval:
    • Adobe sold the 3D PDF (PRC) business last year and to TechSoft.
    • Adobe Acrobat V10 will no longer support the CAD translations.
    • Companies like Boeing are migrating from Adobe 3D PDF to STEP for LOTAR.
    • The AIA (Aerospace Industries Association) LOTAR standard is STEP
    • Government organizations like the FAA require STEP for long term archiving of all civil aviation 3D data.
    • ALL FAA 3D data requires third party Validation, so you have to verify and check your archive content by validation. You cannot move the data to any format without validating the data is the same in the new format. All backup systems have validation (verify check) to make sure the data is the same.
    • Kubotek developed an independent tool, Kubotek Validation Tool to validate that the data is identical, once it has been moved to a new format. The Kubotek Validation Tool checks the Geometry and many other parameters including PMI.

  • Anonymous

    More info on LOTAR for your reading pleasure:

    http://www.slideshare.net/zurayrs/LOTAR101OverviewCurrentJan2009

    Scott

  • http://twitter.com/bcourter Blake Courter

    D-

    I think you might want to dig into JT a little more. My understanding is that it has received approval as a standard (ISO, I think), although I don’t know whether there is a second implementation. (SpaceClaim also uses the JTopen libraries.)

    I can tell you that we have a lot of customers who use JT as their archival and visualization format, along with SpaceClaim to do what-if studies and make new concepts using that data. I’ve heard a lot vendors talk about their vision for 3D collaboration, but I’ve been most impressed with customers who use JT. It just plain works, and kudos to the JTOpen team for delivering a platform on which third parties like us can also stand.

    Here’s an example workflow I’ve seen at many times. Some engineers are doing preliminary work on a new vehicle program. They go into VisMockUp and take a look at the entire vehicle in JT. They save out some subassemblies in an area they’d like to improve. They read that JT into SpaceClaim, where they swap in new components from other engineering teams and suppliers, and the redesign as necessary to understand the impact of that change. Then they save it out of SpaceClaim as JT and bring that back into the entire vehicle mockup. When done, they release the JT to the CAD team as the specification for detailed design.

    It’s beautiful: JT enables a true multi-CAD environment. And it’s not just for NX shops – I’ve seen it with Pro/E and v5 – over and over JT just plain works. No wonder 20+ automotive companies have agreed with each other to standardize on it. It wouldn’t surprise me if JT support was a factor in Daimler’s recent decision to change primary CAD tools. The business benefits of equipping every engineer with the ability to create and communicate in 3D are overwhelmingly compelling.

    -Blake
    (A SpaceClaim marketing guy)

  • http://www.deelip.com Deelip Menezes

    I get your point about LOTAR and STEP. But the fact that Adobe no longer single handedly controls PDF is a good thing, which was my point. I’d like to see STEP improved and maintained more than anyone else. What I’d not like to see is STEP going down the path of IGES.

  • http://www.deelip.com Deelip Menezes

    I think you are missing my point. I don’t think anyone is saying JT does not work or does not do a good job. Its fantastic. But who exactly controls JT? ISO or Siemens PLM?

  • http://twitter.com/bcourter Blake Courter

    It’s not uncommon to have a single vendor push for their proprietary standard to be ratified. Who is driving 3D PDF? Our friends at Tech Soft? Surely not ISO.

    It’s also interesting to ask how important ISO is. I don’t have a firm opinion, but if you can call JTOpen, get a copy of the spec, and implement it yourself, does it matter? Even if you still need to license Parasolid to make it practical?

    Do you know if there’s an alternative to JTOpen?

    -B

  • http://www.deelip.com Deelip Menezes

    From http://www.iso.org/iso/pressrelease.htm?refid=Ref1141

    “The Portable Document Format (PDF), undeniably one of the most commonly used formats for electronic documents, is now accessible as an ISO International Standard – ISO 32000-1. This move follows a decision by Adobe Systems Incorporated, original developer and copyright owner of the format, to relinquish control to ISO, who is now in charge of publishing the specifications for the current version (1.7) and for updating and developing future versions.”

    The keywords are “relinquish control to ISO” and “updating and developing future versions”. Does JT has that kind of a system in place with ISO?

    And are you suggesting that I need to license Parasolid (a proprietary technology) to implement JT (a supposedly open standard)?

    As regarding how important is ISO? As important as the word “Standard” I would assume. ;-)

  • http://soliddna.wordpress.com/ Solid DNA

    I agree with Blake…What difference does it make? ….as long as other companies decided to use it.

    I think that Siemens has prove it, with Parasolid, Dcube, and probably other technologies and now JT that they can be more open than any other CAD company.

    I read in a blog “…Siemens PLM made it clear years ago that “open” is their mantra….it seems to be working…”

  • http://www.deelip.com Deelip Menezes

    Well, if it does not make any difference whether a file format is controlled by a single company or not, then why not use the native file format of the CAD system itself for long term data storage? Why do we need all these neutral formats?

  • Phil

    I don’t believe that Boeing is moving away from 3D PDF for it’s initiatives. I wonder where you get this information?

  • http://twitter.com/thomasteger Thomas Teger

    Deelip is right on with his comment about JT. It is truly a fantastic way of exchanging data. The quality is awesome.

    But the thing I never understood is the JT Open Program. If it is in fact “a standard”, then why is it locked up, and I as a solution provider have to license it? The licensing fee is based on the price of the product I am selling which also forces me to introduce further complexity to my product offerings. ISO or not it is a PITA.

  • d3print

    I agree with that Deelip! The standards give us the guidelines, it doesn`t matter is it ASME or ISO or whatelse.
    I think JT will be standard for CAD format.

    Thanks,
    d3

  • d3print

    I agree with that Deelip! The standards give us the guidelines, it doesn`t matter is it ASME or ISO or whatelse.
    I think JT will be standard for CAD format.

    Thanks,
    d3

  • http://twitter.com/jmerryhaus Jim Merry

    Hi Blake/Deelip – the ISO imprimatur matters quite a bit to companies with long views, e.g., 60+ years. JT has been designated a “Publicly Available Specification” by ISO which is in actuality completely meaningless. More material is that Siemens have taken the step to push JT through the complete ISO process and it has been accepted as a ‘New Work Item’ under the STEP committee. In addition to access to the spec, ISO also ensures a public and open process for evolution of the spec as Deelip notes.

    To clarify a bit on the 3D PDF side: PDF 1.7 was released to ISO and became ISO 32000:1 and it does not include the PRC specification. That was part of PDF 1.71. PRC was accepted as a new work item by ISO, but under the PDF-E committee and is about halfway through that process. Adobe participates, Techsoft participates and so do several other companies including Boeing — so it is being driven forward in an open and public manner by a group of interested stakeholders. PDF-E, PDF-A, PDF-X, and PDF-H will all be folded back into ISO 32000:2.

    Also – someone mentioned STEP 203 E2 and the fact that CAD companies are not rushing to support it. What is nice about the availability of the standard is that 3rd parties like Techsoft, coretech, datakit, etc can extract the CAD data and write out STEP 203 E2 and as long as someone like Kubotek, Elysium or ITI validates that the translation is accurate, you have a great way to vault data for long term archival/retrieval.

  • http://www.deelip.com Deelip Menezes

    A quick question. Does a developer need to license Parasolid to read XT Brep (not JT Brep) information stored in a JT file?

  • http://twitter.com/AndyKS AndyKS

    Deelip,

    No, you do not need a Parasolid license to read or write the XTBrep representation to or from a JT file. The JT Open Toolkit has routines that will accept or return the Brep information in data structures that can be used without requiring Parasolid.

    The XT format specification is also included in the JT file format specification for completeness. So even if you choose to work from the file format definition alone you do not need a Parasolid license … but it obvious creates more demands on the application to try to rationally process the information – hence the JT Open and Parasolid toolkits.

  • http://www.deelip.com Deelip Menezes

    Yes, I see that the XT format spec is listed as Appendix F in the JT spec. That is good to know.




Archives

© 2014 Deelip.com. All Rights Reserved. Deelip.com is a registered trademark of Deelip Menezes. Log in - Designed by Gabfire Themes