It is currently Wed Nov 22, 2017 10:28 pm

All times are UTC - 5 hours




 Page 1 of 1 [ 18 posts ] 
Author Message
 Post subject:
PostPosted: Wed May 30, 2007 9:03 am 
Site Admin

Joined: Sun Feb 11, 2007 8:59 am
Posts: 1105
Location: Ontario Canada
VMK 32A/B introduces you to the Caligari File Format which we will be parsing. Caligari files can be saved as either ASCII (text) files or binary files. We will only be creating a parser for the binary file format.


Offline
 Profile  
 
 Post subject: Re: GameDev VMK 32 - Parsing Binary Files
PostPosted: Tue Jun 12, 2007 6:15 pm 
Site Admin

Joined: Sun Feb 11, 2007 8:59 am
Posts: 1105
Location: Ontario Canada
VMK 32C/D/E shows you how to parse the PolH, Mat1 and ShBx chunks contained in Caligari COB or SCN files. Data that we are interested in is saved into variables and displayed to the screen.

In the Parse function at the top where we do this:

(strncmp(fileHeader.cEndian, "LH", 1) != 0)

There is a little typo here. The 1 should actually be a 2, but the code will still work because the two possible Endian values are LH and HL which differ at the first character.


Last edited by Marek on Thu Oct 09, 2008 6:33 am, edited 1 time in total.

Offline
 Profile  
 
 Post subject: Re: GameDev VMK 32 - Parsing Binary Files
PostPosted: Thu Jun 14, 2007 6:28 pm 
Site Admin

Joined: Sun Feb 11, 2007 8:59 am
Posts: 1105
Location: Ontario Canada
VMK32F/G takes the parser project developed in VMK32C/D/E and moves it over in to the game engine. A new class called CaligariFile is created that allows you to load in all the data from a Caligari file and store it in memory for use.


Offline
 Profile  
 
 Post subject: Re: GameDev VMK 32 - Parsing Binary Files
PostPosted: Mon Jun 18, 2007 7:39 am 
Site Admin

Joined: Sun Feb 11, 2007 8:59 am
Posts: 1105
Location: Ontario Canada
VMK32H shows you how to implement the CaligariMat1 class used to store all the Material chunk data in memory.


Offline
 Profile  
 
 Post subject: Re: GameDev VMK 32 - Parsing Binary Files
PostPosted: Fri Jun 22, 2007 6:01 am 
Site Admin

Joined: Sun Feb 11, 2007 8:59 am
Posts: 1105
Location: Ontario Canada
VMK32I shows you how to implement the CaligariShBx class. I use the parser project that we finished in VMK32E to analyze the ShBx chunk before I show you how to implement the SaveParameter function inside CaligariShBx. Near the end of the VMK I also clean up some code to get ready for the big build.


Offline
 Profile  
 
 Post subject: Re: GameDev VMK 32 - Parsing Binary Files
PostPosted: Tue Jun 26, 2007 5:50 pm 
Site Admin

Joined: Sun Feb 11, 2007 8:59 am
Posts: 1105
Location: Ontario Canada
VMK32J shows you how to implement three new classes. CaligariPolH, Vector2 and GeometryCompiled. In VMK32H I forgot to implement the SetColor function so I go and show you how to do that in this video.


Offline
 Profile  
 
 Post subject: Re: GameDev VMK 32 - Parsing Binary Files
PostPosted: Sun Jul 01, 2007 9:06 am 
Site Admin

Joined: Sun Feb 11, 2007 8:59 am
Posts: 1105
Location: Ontario Canada
VMK32K/L/M finally shows you how to render a Caligari file that has been loaded into memory.


Offline
 Profile  
 
 Post subject: Re: GameDev VMK 32 - Parsing Binary Files
PostPosted: Wed Jul 04, 2007 11:41 am 
Site Admin

Joined: Sun Feb 11, 2007 8:59 am
Posts: 1105
Location: Ontario Canada
The last set of video's in this mini series (VMK32N/O) show you how to calculate the normals for your loaded shapes so that the surfaces look smooth rather than sharp. The Vector3 class is updated to include the Dot product calculation and some other functions, have a look at Math VMK 7 if you would like a refresher on the dot product.


Offline
 Profile  
 
 Post subject: Re: GameDev VMK 32 - Parsing Binary Files
PostPosted: Tue Aug 07, 2007 5:17 pm 
Site Admin

Joined: Sun Feb 11, 2007 8:59 am
Posts: 1105
Location: Ontario Canada
I've noticed that I started getting a little sloppy with some of the destructors in the Caligari classes. Make sure that all destructors have the word virtual in front of them. I noticed that I forgot to add virtual in front of Vertex2, CaligariShBx, CaligariPolH, CaligariMat1 and the CaligariFile class destructor.



Also you should really convert m_pErrorHandler found in CaligariPolH and CaligariFile to be static s_pErrorHandler as done with all other classes that use the ErrorHandler class.


Offline
 Profile  
 
 Post subject:
PostPosted: Thu Oct 09, 2008 4:28 am 

Joined: Sat Jun 23, 2007 7:56 pm
Posts: 145
Due to the recent release of TrueSpace 7.6 as a free tool, many of us will switch over to using it for our COB files.

I recently did that and noticed that the MAT1 Chuck has been modified for Objects saved as "Caligari Object *.cob" - (The very first file type when you choose Save/Save As)

The header version is major = 0 and minor = 8.
This Mat1 chunk is the same as version 0.7 with an extra FLOAT before the flag to show if we have an Environment, Texture or Bumpmap section.

the needed chage is - Just before the part that reads the 2 bytes (flag) which might contain 'e:' , 't:' or 'b:'

add
if( mat1.chunk.minor_verion == 8 ) {
   //read in one float for the new KD index
   fread(value, 1, sizeof(float))
}


//code continues as usual i.e read 2bytes and check if we have maps


Offline
 Profile  
 
 Post subject: How to keep bin parsing syncronized
PostPosted: Tue Feb 17, 2009 3:29 pm 

Joined: Wed Aug 06, 2008 7:53 pm
Posts: 182
Location: Russia
Hi guys.

I have discovered a useful trick that can be helpful whenever you parse a chunk based binary file, such as Caligari COB/SCN.

A common problem that may arise during the parsing is that the format specifications can be obsolete or incorrect in tiny details and consequently your function that is parsing a current chunk may read too little or too much data. As a result the whole reading process is going out of synchronization, and the function can't parse correctly the rest of the file, since it can't find the beginning of the next chunk.
However, chunk based files usually have a "size" field using which you can harmlessly skip the chunks you are not interested in (as it was said in these VMKs). I offer to use this "size" field to check/correct current file pointer at the end of the chunk parsing procedure to maintain the synchronization.

inline void ParserCob::StoreOffset()
{
    m_fpos = ::ftell( m_file );
}

bool ParserCob::CheckOffset()
{
    if ( !m_chunkHeader.numBytes )
    {
        // Nothing to do since the size is unknown.
        return true;
    }

    const long cur_pos = ::ftell( m_file );
    const long ideal_pos = m_fpos + m_chunkHeader.numBytes;

    if ( ideal_pos != cur_pos )
    {
        const long difference = ideal_pos - cur_pos;

        Log( LogModeBasic, _T( "WARNING! Current FP is wrong (fix the parser). FP was updated." ) );
        Log( LogModeVerbose, _T( "Cur. FP: %d, must be: %d, delta: %d (%s)." ),
            cur_pos,
            ideal_pos,
            difference,
            ( ( difference > 0 ) ? _T( "UNDERFLOW" ) : _T( "OVERFLOW" ))
            );

        ::fseek( m_file, ideal_pos, SEEK_SET );

        return false;
    }

    return true;
}



At the beginning of the parsing function you need to read the chunk header with the "size" field and then store current file pointer:
ReadChunkHeader();
SaveOffset();

At the end of chunk parsing you check whether current file position is correct:
CheckOffset();
If offset is wrong, this function will correct it. So, even if you has read only a half of the chunk, the whole parsing process won't be jammed.


Offline
 Profile  
 
 Post subject: Just in time
PostPosted: Tue Feb 17, 2009 3:43 pm 

Joined: Wed Aug 06, 2008 7:53 pm
Posts: 182
Location: Russia
codeslasher wrote:
... The header version is major = 0 and minor = 8.
This Mat1 chunk is the same as version 0.7 with an extra FLOAT before the flag to show if we have an Environment, Texture or Bumpmap section ...

Thanks for this consecrated info! JIT. :!:


Offline
 Profile  
 
 Post subject: Something about auto_ptr
PostPosted: Sun Mar 08, 2009 12:05 pm 

Joined: Wed Aug 06, 2008 7:53 pm
Posts: 182
Location: Russia
"We are not gonna render holes, 'cause holes are holes"! (c) Marek. :)

Something around VMK 32 F/G (alas, I'm still here :oops:) following repetitive pattern can be found (a pseudocode):
{
    CaligariPolH * polH = new CaligariPolH;

    (...)

    // many similar checks...
    if ( some_error_check ) {
        // error detected => cleanup and exit
        delete polH;
        return false;
    }

    (...)

    m_vPolH.push_back( polH );

}


This pattern was repeated in several places with several different types. Pretty boring, innit? So my suggestion is to use std::auto_ptr<>. In this case you do not need to delete the object explicitly again and again in case of any error. The auto_ptr will delete the object automatically, as soon as the control flow will leave function scope. Instead of std::auto_ptr<> we can use other similar smart pointers, for instance from boost library.

Let's reimplement the fragment above using std::auto_ptr<>.

#include <memory> // add this in <stdafx.h>

{
    std::auto_ptr< CaligariPolH > polH ( new CaligariPolH );

    (...)

    if ( some_error_check ) {
        // error detected => cleanup and exit
        //  delete polH; // <-- you must not delete this obj explicitly
        return false;
    }

    (...)

    m_vPolH.push_back( polH.get() );

    polH.release(); // to prevent obj destruction since now the owner is m_vPolH
}



:roll:



_________________
«Computer scientists deal with algorithms that you may call practical in theory but unpractical in practice.» © Timothy Gowers
Offline
 Profile  
 
 Post subject: VMK 32. Something about Caligari files rendering.
PostPosted: Mon Mar 23, 2009 5:33 pm 

Joined: Wed Aug 06, 2008 7:53 pm
Posts: 182
Location: Russia
These Video Modules of Knowledge made me really happy. O-la-la! At last we can download and render complicated meshes!


A few Q & CC (case comments).

0). Some parts were not clear for me until the whole code for analysis is available. Because of that I felt sometimes that I do not understand what is going on. So, at the time of typing (especially when one does not repeat the code from VMK one for one) I would prefer to had a little bit more explanations of the algorithms and data structures that were used. Now it is all over (for me), but my opinion is that 5 more minutes of video with explanations of the solution (especially on the preparing to the rendering process, i.e. what for the structure MaterialAngle, etc) might have been quite useful. :roll:


1). Following piece of code removes unused materials. Testing the code I have downloaded almost all objects from standard Caligari library and never met this condition (the removal of extra material).

Image

My question is - why this code was incorporated at all?



2). A minor hint. :roll: Here is a piece of code from the smoothing algo:

Image

This code can be replaced with just one line:
m_vVertexFaces.resize( m_vVertex.size() );

I know that Marek knows, ( that I know, ) but for those who still don’t know... :)


In general, I like these VMKs => there are several good engineering solutions there, such as the way how Caligari co-ordinate space is converted into GE one (just by transposing vector-columns of the position matrix) or how the number of faces per vertex are calculated in the smoothing algo (using one cycle of linear-complexity)!

:roll:


Offline
 Profile  
 
 Post subject: Why some materials seem missing
PostPosted: Mon Mar 23, 2009 5:52 pm 

Joined: Wed Aug 06, 2008 7:53 pm
Posts: 182
Location: Russia
It is possible that when the GE is loading some certain Caligari files some materials are reported being missed. A small investigation shown, that the reason is that in such cases the chunks Mat1 are replaced with the chunks PrTx (PROCEDURAL TEXTURE CHUNK). Our COB loader does not process PrTx yet (consider this as an exercise). 8)

Caligari trueSpace file format wrote:
'PrTx' (Procedural Texture Chunk)

The procedural texture chunk is very similar to the material chunk. If a material is to be
procedurally textured, then this chunk will appear instead of a 'Mat1' chunk. The only
difference between the two chunks is that where a 'Mat1' has an optional texture map
specification, the 'PrTx' chunk has an optional Granite, Marble or Wood texture specification.
Other than this, everything else is the same.


Last edited by BugHunter on Sun Mar 29, 2009 4:42 am, edited 1 time in total.


_________________
«Computer scientists deal with algorithms that you may call practical in theory but unpractical in practice.» © Timothy Gowers
Offline
 Profile  
 
 Post subject: Re: VMK 32. Something about Caligari files rendering.
PostPosted: Tue Mar 24, 2009 7:39 am 
Site Admin

Joined: Sun Feb 11, 2007 8:59 am
Posts: 1105
Location: Ontario Canada
BugHunter wrote:
1). Following piece of code removes unused materials. ....
My question is - why this code was incorporated at all?


When I created the Caligari parser I was using trueSpace v6.0 and I noticed that sometimes when I created complex scenes, I would have materials saved to my file that were not used by any of my geometry anymore. Maybe this was a bug in the program I don't know. I haven't used trueSpace in a while so it might not be an issue anymore with v7.6


Offline
 Profile  
 
 Post subject: Ca 6.0 vs Ca 7.6 - this makes sense
PostPosted: Tue Mar 24, 2009 9:41 am 

Joined: Wed Aug 06, 2008 7:53 pm
Posts: 182
Location: Russia
Ok, this makes sense, thank you.

“In the process of the design, many issues were faced, and some choices were made, that might not be intuitively right. But in each case they were taken for some reasons.â€


Offline
 Profile  
 
 Post subject: Re: GameDev VMK 32 - Parsing Binary Files
PostPosted: Fri Oct 07, 2011 5:38 pm 

Joined: Sat Aug 16, 2008 7:58 am
Posts: 449
I took the code from the VMK where we create a seperate cmd project and added the feature of printing the information into a text file. This way if you have a large file you are reading in, where your console wont allow you to go back to the beginning, you can see all the data that belongs to this file. Yes you could just save the .cob or .scn file as a ASCII from truespace, but it has all the other information that we are not interested in, now you can see the data that pertains to just the GameEngine.

If you have been following these videos and have successfully integrated the parser into the GameEninge and you load in the mech.cob file. It looks complex, but yet simple enough, because the main rule for parsing the data is that the faces have to be triangulated, the stand alone parser generated about 27,000 lines of information on it. This is without all the other features from truespace that we skipped over, we only save about 8-10% of the details. I will provide a copy of the file here, but I will put it inside of a code block since it is so long.

My saved mech.txt file.
...

I was going to put the txt in this post inside of a code block, but the maximum # of characters is 25 or 50k and the file generated oh about 500k characters. LOL

So if you would like a copy of the txt file, just reply to this or send me a message.


Offline
 Profile  
 
Display posts from previous:  Sort by  
 Page 1 of 1 [ 18 posts ] 

All times are UTC - 5 hours


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Jump to:  

cron