It is currently Tue Dec 12, 2017 10:21 pm

All times are UTC - 5 hours




 Page 1 of 1 [ 11 posts ] 
Author Message
 Post subject:
PostPosted: Wed May 02, 2007 6:49 am 
Site Admin

Joined: Sun Feb 11, 2007 8:59 am
Posts: 1105
Location: Ontario Canada
In VMK31A I outline how the text parser will work and what are the keywords associated with the parser.


Offline
 Profile  
 
 Post subject: Re: GameDev VMK 31 - Parsing Text Files
PostPosted: Sun May 06, 2007 7:34 am 
Site Admin

Joined: Sun Feb 11, 2007 8:59 am
Posts: 1105
Location: Ontario Canada
The *.lvl file parser is created in VMK31B/C/D. A lot of string/text manipulation is used to identify keywords and options in the file and the results are displayed on the screen. Use microsoft's MSDN library (msdn.microsoft.com) to understand how different functions work.



As I was watching these VMK's I noticed that I mis-spelt Parser when I was typing in the file name for the cpp file. Don't worry about this too much, seeing as in the next VMK we are going to be porting all this code into our game engine.


Offline
 Profile  
 
 Post subject: Re: GameDev VMK 31 - Parsing Text Files
PostPosted: Fri May 25, 2007 7:59 am 
Site Admin

Joined: Sun Feb 11, 2007 8:59 am
Posts: 1105
Location: Ontario Canada
The *.lvl parser created in the previous VMK is ported over into the game engine in VMK31E/F/G/H. There is a lot of repetitive code shown in part E,F and G. Feel free to skim over the content. MAKE SURE you watch the entire VMK31H through because I go through the process of fixing a bunch of bugs that have crept up on us over the past couple of VMK's.


Offline
 Profile  
 
 Post subject: Re: GameDev VMK 31 - Parsing Text Files
PostPosted: Sun May 27, 2007 1:08 pm 
Site Admin

Joined: Sun Feb 11, 2007 8:59 am
Posts: 1105
Location: Ontario Canada
if you did delete House.h and Triangle.h from your project make sure you delete any includes in your code that may reference to these no longer needed header files.


Offline
 Profile  
 
 Post subject: Bug found in parser
PostPosted: Tue Aug 26, 2008 8:09 am 
Site Admin

Joined: Sun Feb 11, 2007 8:59 am
Posts: 1105
Location: Ontario Canada
chester30 has found a bad bug in the parser. Luckily it is easy to fix.

inside Scene::findKeyword under the RETURN_CHAR section you will see a call to memset to clear the variable szName. The problem is that the compiler has no way to know how big this variable is! To fix this, lets make szName always be 256 characters long. Now you can change this line of code to be:
memset(szName,0,256);


If you look through all the strings that you can parse, they are all set to be 256 characters long, all except for one. That one is inside the Light section. Inside the Scene::ParseLine function in the section that handles lights, change the szType variable to be 256 characters long like so:

else if (strncmp(&szDataLower[iCount],"light",5)==0) {
   //convert the rest of the string after "Light " to lower case and start searching for keywords

   //variables needed to create a Light node      
   char  szName[256];
   char  szType[256];

   [all the rest is the same]


I have uploaded a video herethat shows the changes that need to be made.


Offline
 Profile  
 
 Post subject: Invalid options
PostPosted: Mon Jan 05, 2009 5:17 pm 

Joined: Wed Dec 24, 2008 4:25 pm
Posts: 16
Is it intentional that invalid options are ignored silently? The effect is that a misspelled option is ignored and the defaults apply. This can result in a level that is awfully hard to debug. I don't have a fix (yet) but it seems important.


Offline
 Profile  
 
 Post subject: Misspelt vs Defaults (ps. Let there be head-light at[0])
PostPosted: Tue Jan 06, 2009 2:32 am 

Joined: Wed Aug 06, 2008 7:53 pm
Posts: 182
Location: Russia
[1] First, there is no connection between anything misspelt and the defaults, because you can’t know in advance, whether the misspelled word denotes an allowed option or it is just an alien artifact.
My version of the parser, among many other enhancements, has the flag “LogMode”, which has 4 possible values: ~ Silent (log no messages whatsoever), Basic (log only errors), Normal and Verbose. So, depending on the state of this flag GE-logger reports or ignores different events during the parsing procedure. Btw, similar approach you can meet everywhere in the software industry, i.e. VC has “Warning level” setting, MS ROTOR project (.NET runtime sources, available for free) has similar machinery built-in, etc. etc.

[2] Second. My opinion about defaults is that they should not be governed by parser, but be defined by default states of creating objects. The constructors of almost all types in GE (geometry, materials, texture transformation etc) initialize the objects with such default values, therefore there is no need to duplicate their work. It is enough that default values are described in the LVL-format documentation (pdf), which is important for potential level-designers, who theoretically know nothing about internals of GE.

I’m working hardly on these VMKs (“Parsing Text Files”) and have a lot to say. Hope a little later I will add more comments.

PS. It’s a good idea to add unrecognized word in the log. Thanks.
_______________________________________
Happy New Year - don't forget - it is 2009 now!



_________________
«Computer scientists deal with algorithms that you may call practical in theory but unpractical in practice.» © Timothy Gowers
Offline
 Profile  
 
 Post subject: Misspelt vs Defaults
PostPosted: Tue Jan 06, 2009 10:32 am 

Joined: Wed Dec 24, 2008 4:25 pm
Posts: 16
I understood that defaults are not handled by the parser; when I said "defaults apply" my intent was that what ever defaults were built into the object constructor would apply, since nothing would be done about them by the parser.

After completing the Parser construction last night, I thought about whether I wanted to go ahead and reimplement the kind of check that I'm concerned about, and decided to defer that until later. I have saved a pseudo code description of it for reference at that point.


Offline
 Profile  
 
 Post subject: ctor’s defaults almost useless during parsing
PostPosted: Tue Jan 06, 2009 3:42 pm 

Joined: Wed Aug 06, 2008 7:53 pm
Posts: 182
Location: Russia
Oh, my apologies :oops: As it turned out there is possibility to some of (non-required) parameters be missed, whereby we have to use externally defined default values (as described in LVL-doc and as it was done in VMK31) if level-file has not provided such options. Basically the only situation when constructor's defaults can be used efficiently is when there are no options at all (except required), i.e.
Box GeoID_ 2009

But it is impractical to discriminate such rare cases.


Offline
 Profile  
 
 Post subject: a link
PostPosted: Wed Jan 14, 2009 8:03 pm 

Joined: Wed Aug 06, 2008 7:53 pm
Posts: 182
Location: Russia
A crosslink to the LVL files thread
http://www.marek-knows.com/phpBB3/viewt ... =1408#1408


Offline
 Profile  
 
 Post subject: LVL-format corrections & extensions.
PostPosted: Wed Jan 21, 2009 7:14 pm 

Joined: Wed Aug 06, 2008 7:53 pm
Posts: 182
Location: Russia
LVL-format corrections & extensions


FIX

There is a couple of typos in the PDF file describing LVL format, namely, underscore character '_' is missing of:
• Keyword "Shape" Option "TexTfmID" => "TexTfmID_"
• Keyword "TexTfm" Option "TexTfmID" => "TexTfmID_"

All option-tokens have the character '_' as a terminating symbol (terminal). We can use this fact to extract them from the parsed line (although I would prefer to see the space character as a terminal). This also allows us to use another algorithm to extract option-tokens. Marek uses "find" approach, which means "multipass" algorithm, - one should scan the parsing line N times, where N is the quantity of options for given Keyword. E.g., "Box" has 4 options: "Dep_", "GeoID_", "Hei_" and "Wid_". Thus "multipass" algo scans the line 4 times. Another way to parse the line - "single-pass", when one extracts the tokens from the input stream one by one, and then determines token's type by context and/or using look-up keyword-table. ("Single-pass" approach is usually more efficient etc.)


EXTENSIONS

Note: each of proposed extension is implemented in my version of compiler; therefore they are realizable in principle.

First of all, I reckon that all extensions should be backward compatible, e.g. the parser should be able to read the files in both formats - original and extended.

Another important issue relates to the main property of text-format, namely, it is meant to be human readable and human friendly. Most of my extensions make the LVL format more tolerant towards human errors.

1) Let's make the LVL-format be more tolerant towards WHITE SPACES (space and tab characters). One can put white-space character to separate any keyword, option-keyword and parameters. Leading spaces should be acceptable as well. For instance, following line should be considered as being correct:

  Box   GeoID_ 2009  Wid_ 1  Hei_ 2  Dep_ 3

To relieve the task I suggest to add the function SkipWhiteSpaces().

2) Full fledged C++ style comments support. Yes, it is a good exercise to implement full support of the commenting in your text file:
2.a) full support of single-line comment. For example, following lines should be considered as equivalent:

Box GeoID_ 2009 Wid_ 1 Hei_ 2 // Dep_ 3

Box GeoID_ 2009 Wid_ 1 Hei_ 2

2.b) full support of multiline C-style comments. A hint - the internal comments of the parsed line could be replaced with spaces, then if you implemented 1) (white-space tolerance) those white spaces will be skipped by the parser.
Example:

  Box   GeoID_ /* 2009 */ 2010 Wid_ 1  /* Hei_ 2 */ Dep_ 3 Hei_ 0.5

could be treated as:
  Box   GeoID_            2010 Wid_ 1               Dep_ 3 Hei_ 0.5


Some of the ideas how to handle multi-line comments you can borrow from here. However, multiline commenting also means that you can start the comment in the middle of some line and then close it in the middle of one of the next lines. Remember also about comment's priority, i.e.:

Box GeoID_ /* 2009 */ 2010 Wid_ 1 /* Hei_ 2 Dep_ 3 Hei_ 0.5
Box GeoID_ /* 2009 2010 Wid_ 1 Hei_ 2 Dep_ 3 Hei_ 0.5
// Grid GeoID_ 2011 */ Box GeoID_ /* 2009 */ 2012 Wid_ 1 Hei_ 2 // Dep_ 3 Hei_ 0.5

Should be treated as:

Box GeoID_ 2010 Wid_ 1
Box GeoID_ 2012 Wid_ 1 Hei_ 2

IOW try to implement all properties of the comments you can find in C++ commenting style.

3) Add support of the strings with internal spaces. To allow space char in the name, i.e. in option Shape Name_ or Texture File_, put the name in double quotation marks, like this:

Texture File_ "textures/Spaceship Cafeteria.tga"

For backward compatibility the old format (without quotation marks) should be supported as it had done before.

4) Multiline strings. If the line is terminated by '\' (back slash) character, then is is considered as multiline and the content of the next line should be concatinated to the current line. Example:

   +++ Shape \
       GeoID_ 766 \
       MatID_ 667 \
       Name_ "man of war" \
       TexID_ 321 \
       TexTfmID_ 123

This line is convertible to
+++ Shape GeoID_ 555 MatID_ 667 Name_ "man of war" TexID_ 321 TexTfmID_ 123

5) Add the option Vis_ (visibility) to the Shape (shape node), similar to Transform Vis_:
type = int value, meaning: 0 = invisible, 1 = visible. Default value = 1
See the discussion here.

6) Parsing structure: try to separate the host code (the Parser) and the client part (SceneBuilder). Let the parser passes the parsed lines (in some internal format) one by one to the SceneBuilder. An advantage of this approach is that if you manage to implement this you can later on seamlessly add more and more formats (host parsers), i.e. XML, binary, and any compatible proprietary formats without the necessity to rewrite the code, which creates the scene.

7) Add rollback feature

8 ) The last but not least feature I would propose is the MUTATOR, or the extension applicable to the options' parameters describing some simple linear fluctuations. LVL-code might look like this:

+ Transform Trans_ 0, (0.0 1.0 0.0 0.01), (0.0 -1.0 0 0.03) Axis_ 0 1 0 Angle_ (-90.0 +90.0 0 1)

The format of MUTATOR is (MIN, MAX, CUR, STEP, ID). ID is optional.
As could be seen if one of the parameters of some option is enclosed by round brackets then this parameter has special meaning. This parameter is not just the constant any more, but a value, which fluctuates between MIN and MAX. STEP is the value which is added to CUR value periodically (say each 10 ms). Almost any float point parameters could be mutated by MUTATOR. I used Wink class as a good helper.

More complete explanation of MUTATORS (with working exe) needs a separate article that I am going to write as soon as possible.



_________________
«Computer scientists deal with algorithms that you may call practical in theory but unpractical in practice.» © Timothy Gowers
Offline
 Profile  
 
Display posts from previous:  Sort by  
 Page 1 of 1 [ 11 posts ] 

All times are UTC - 5 hours


Who is online

Users browsing this forum: No registered users and 0 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum

Jump to:  

cron