I read the pdf file you wrote, but unfortunately the truth is that it didn't help much. For me, random characters and structure data are not quite what I need. Instead of saying "hey, this is a 4 character data", I am more interested in the "Hey, this 4 character data represents feature X in a real model". My problem is that I can't link the Hex information with the real world models. And google is no longer enough :S
"Hey, this 4 character data represents feature X in a real model"
Blender python script does this for you, if you understand python code you understand where something is stored in hex.
I give example (could someone upload sc2 default cube.m3? [plane or triangle would be even better, vertex.m3 would be pure awsomeness]). Export cube to Blender with python script. Go to UV editor (how ever it is called), pick one UV coordinate. Apply on its coordinates wisdom:
From m3lib:
Note: Important that you read in UV coordinates as signed short values
realUV = 1771 / 2048.0
realUV = 0.8647
1771 in HEX is 6eb (1771 might be UV's U or V, it does not matter for example). Search from m3 file 06eb (or be60 if it is stored reverse) and you will find:
feature X in a real model
Then to further evolve this theory (from m3lib, formatting is thx to forum formatting scrambled, how do I add tags?):
struct Vertex32 32 byte
{
Vec3D pos;
char boneWeight[4];
char boneIndex[4]; index in boneLookup of vertex submesh
char normal[4]; x, y, z, w (w is the scale)
int16 uv[2];
char tangents[4];
};
char is 1 byte, char boneWeight[4] means 1 byte x 4 = 4 bytes, that makes 16 char bytes total.
int16 is 2 bytes, int16 uv[2] means 2 bytes x 2 = 4 bytes.
16 + 4 bytes = 20 bytes
struct Vertex32 32 byte, but where are the missing 32 - 20 = 12 bytes? I am using C variable types as base. In HEX file I think you always search for cluster of bytes. Because byte on its own can not give info.
I double post instead editing to a) bump thread b) this is good discovery for me to share.
I wrote to pdf file how to understand what you see on http://code.google.com/p/libm3/wiki/MODL page or any libm3 page. After reading this you should look different way on libm3 writings. It is like you almost can read the code.
PDF is attached as attachment.
Made wiki for m3 export scripts. Registration needed at GitHub (takes only seconds[TM]). Link to project: https://github.com/InDesign/M3TOX3D
Paste that link to your signature and tell a friend! Everyone are welcomed from sc2mapster forums to involve. Spread the word.
EDIT:
I added to wiki all functions (both Blender and 3ds Max). There are currently only importer functions (you really do not need exporter functions because when you can read out, you can reverse that process to write exporter). Next task is to write tutorial how community can participate in this project.
I lack skills to offer help to you DrSuperEvil. I read about sc2 Diablo thread. There I learned what is m3a format: a m3 file storing only animations (animated bones).
In sc2 Diablo thread had some good suggestions. I understood it was agreed model_name.m3 uses animations from model_name.m3a. Rough outline would be to import into 3ds Max sc2 Diablo, delete its geometry, animate only skeleton (Diablo bones) export new file as SpaceDiablo.m3a (make link in GalaxyEditor) and look does it work.
Same could be tried with simple cube model to verify process itself is valid. Create cube (triangle or plane) in 3ds Max give it one bone (so geometry would be linked to bone), save as m3. From your saved m3 file delete geometry, animate bone to rotate, save as cube.m3a. Load into GalaxyEditor and look does it work.
I understand goal is to keep sc2 Diablo model as it is and do all the animaton in m3a file because (importing Diablo will lose data because nature of Blizzard close source model format, though 3ds Max import script is best what we have currently).
On thread topic: Considering EULA of SC2 and copyrights its illegal to import data out of SC2 (models). To make it short X3D2M3 is legal (somewhat) while M3TOX3D is illegal for sure. In this light my previous suggestions about Diablo model manipulating are illegal. Still our project here is good for educational, non-profit use. Its one of these things legal to own, illegal to use.
If I get time I will publish tutorials how to join project, learn to help us and share learned findings on wiki page (commenting code). Additional bonus is that if you find our done work $][!", you can always make it better yourself. Just any start is always hard.
Diablo model, I did not know what was m3a and searched forums. Search returned Diablo model thread where I learned what is a m3a format.
@Frizi
I'm really beginner in GitHub. I have no idea what are all these pull requests or how they merge with my repos (though I have a slight assumption). I forgot to add in PM in GitHub to you that moving code tags between text will explode wiki-page, I have no idea why.
At that time I did not know any better solution to creating environment for commenting code. Wiki had history and easy user access possibility. If you manage to make it better post here quick explanation how people can use it (access to comment code). Simplicity and quick access is good.
EDIT START
Frizi how I am able to upload new files to my own repo, currently I get "note about fast-forward" error
in short: https://github.com/InDesign/M3TOX3D/pull/1
How do I accept it and move on with my life?
Meanwhile I recreated in python parts of M3I_Ref struct, I learn how to read binary data
(idea is to port maxcript functions which I need to Python)
http://codepad.org/LY2W7OwF
I added much cool reading about Maxscript to Wiki, check it out.
EDIT END
Im in the phase of learning MXS to understand better Toxicated script parts. Though I find most of script functions pretty self explanatory (I have wc3 jass background mostly, mixed with some python for humanitarian and Blender API calls). Currently puzzles me function parameters and structs like myStruct.myFunc.myMethod.myVar.
In real life currently working (finished high school, went to military, came back and are going to continue studies). I understand you (like most of us) want to make this happen just like that. But I can't focus on my hobby (3d modeling and similar) fulltime.
That is why I consider offering opportunity for community to participate. Maybe there are kids who don't know what to do with their free time? Maybe they
could help us with this by reading our comments (made way) and make it better. I bet some think: "omg, programming!?". I once was a gamer then I read
tutorials about JASS and now I do this.
3dsMax script took one year to make what it is today, SC2 took to be made how long? Our project here? Made by two days? :D
I have thinked about re-creating same functions in different programming language (lets take for example python because I know it somewhat). If you
understand what does function result is, you can transfer its idea to totally different programming language.
Maybe some need to learn learning before they are able to learn :D
Take this example (3ds Max Toxicated script):
fn M3I_Main calls fnM3I_Open
That is one total branch which can easily be recreated in python (I am even able to recreate it). My point is it is possible step by step to export script itself to different language. If we are eliminated all the easy things then there is greater chance that people (let it be whomever) can consentrate all their
think-power to that one hard thing. By many on very specific topic it should be solved in no-time (at least faster when trying to do it alone).
Dunno why people underestimate that power what we could use.
My current goal is to reproduce Toxicated mxs script way of reading tags in m3 file. My really first baby steps in mxs <> python conversion: read 33DM from
Bunker.m3 file with my script (kudos to Toxicated and Muraj).
What I did can be done in two ways - - - with 2 lines (or even 1 line) of code or like I did (lots of lines). By doing I learn and that is what it is all about.
Codepad paste: http://codepad.org/ThZK6is4
I can only suggest others to post their progress too, then it is bit more fun and alive.
Sounds facinating. Still it is the .m3a format that interests most people so they can give their zerglings extra dance moves etc. without having to do a top down approach.
Rollback Post to RevisionRollBack
Contribute to the wiki (Wiki button at top of page) Considered easy altering of the unit textures?
No it isn't.
The .m3 format is what interests the most not .m3a, even if one figured out the .m3a format you'd still need someone to make those "extra dance moves" that so many people seem to want.
Either way with Triceron we've been trying to figure out, after so many tests with blizzard's models and our own created models and m3a we've got nothing really :P
Just the conclusion that the .m3 must likely have a flag or something that indicates it that it requires an .m3a or otherwise it will just ignore any attempts of .m3a you try to shove on them. They can't just be data controlled. (meaning that just making a m3a value for the "required animations" field isn't enough)
An .m3 with a m3a reference always has everything bones, mesh, shaders, effects, weightning, ect but no animations. An .m3a, though doesn't it just has bones, lacks mesh, shaders and everything but has the animations. But the weightning and bones for both files have to be the same.
I'm trying to compare files trying to spot a different between them (besides the ones we know) but I only know so few about programming and I'm doing this comparison by opening them in a hex editor cause obviously we don't have proper modelling tools as of yet.
EDIT 09.09.11
I decided to post about progress once in a month. Again knowledge is freely available at https://github.com/InDesign/M3TOX3D It requires people who want to do something with that knowledge.
EDIT END
Was busy with real life work, going to look into m3 format in upcoming days. Importing mxs structs into python class (python way doing structs) seems pretty strait forward, that means quicker copy-paste of written functions and less manual conversion from one language to another.
Hex file comparison, have not never tried it. That is how they found m3 file format in first place: by automatically parsing all m3 hex files and extracting patterns out of it.
Exciting is that could we be the ones who make those modeling tools available to public.
Maybe reply is better than no reply at all. I have legally aquired 3ds max license through their student program (if you are a student in your local school then you are legit to download their software as long as you have official student status [or something like that]). Anyway I have downloaded 2x 3.3 GB of data from their site (2x 24h hours download time, my connection ~). To find out Windows 7 and 3ds Max 2010 is not compatible?!
I had blue dream filling all the Toxicated script with debug messages about data manipulation which would make recreation of Toxicated scripts in Python easy.
I got legit Windows 7, legit 3ds Max 2010 and they do not get along. My dirty fix to this would be to buy new HDD, install legit Windows XP on it (I can get it by school and Windows partnership) and try my luck on that set (needing new usb stick to transfer my precious software [3ds].
My time to summer start is planned and there isn't much room for script development. To my suprise there is EASY way to download and install LEGALY Autodesk 3d Max on your pc IF you are a student (in ANY school around the world), currently registering and starting 3ds max download is as easy as is making a new email address.
Additionally 3ds Max can be registered as a 30 days trial (if you are desperate trial install + OS format after trial should give you hint)
I write once more the idea I had. Maybe some bright mind does it before me:
1. Download Autodesk 3ds Max 2010 (install as trial)
2. Stack Toxicated code full of debug messages to see how script works on test m3 file (bunker.m3 would be good)
3. Dump all [variable data] gotten data to file and upload for others to see
4. Rebuild similar script in free to use environment (I thoguht Python + Blender 3d program).
I read the pdf file you wrote, but unfortunately the truth is that it didn't help much. For me, random characters and structure data are not quite what I need. Instead of saying "hey, this is a 4 character data", I am more interested in the "Hey, this 4 character data represents feature X in a real model". My problem is that I can't link the Hex information with the real world models. And google is no longer enough :S
Google is no longer enough?
"Hey, this 4 character data represents feature X in a real model"
Blender python script does this for you, if you understand python code you understand where something is stored in hex.
I give example (could someone upload sc2 default cube.m3? [plane or triangle would be even better, vertex.m3 would be pure awsomeness]). Export cube to Blender with python script. Go to UV editor (how ever it is called), pick one UV coordinate. Apply on its coordinates wisdom: From m3lib: Note: Important that you read in UV coordinates as signed short values realUV = 1771 / 2048.0 realUV = 0.8647
1771 in HEX is 6eb (1771 might be UV's U or V, it does not matter for example). Search from m3 file 06eb (or be60 if it is stored reverse) and you will find:
feature X in a real model
Then to further evolve this theory (from m3lib, formatting is thx to forum formatting scrambled, how do I add tags?): struct Vertex32 32 byte { Vec3D pos; char boneWeight[4]; char boneIndex[4]; index in boneLookup of vertex submesh char normal[4]; x, y, z, w (w is the scale) int16 uv[2]; char tangents[4]; };
char is 1 byte, char boneWeight[4] means 1 byte x 4 = 4 bytes, that makes 16 char bytes total. int16 is 2 bytes, int16 uv[2] means 2 bytes x 2 = 4 bytes. 16 + 4 bytes = 20 bytes struct Vertex32 32 byte, but where are the missing 32 - 20 = 12 bytes? I am using C variable types as base. In HEX file I think you always search for cluster of bytes. Because byte on its own can not give info.
EDIT:
For everybody motivation: http://wacco.mveas.com/index.php?entry=18
I double post instead editing to a) bump thread b) this is good discovery for me to share.
I wrote to pdf file how to understand what you see on http://code.google.com/p/libm3/wiki/MODL page or any libm3 page. After reading this you should look different way on libm3 writings. It is like you almost can read the code. PDF is attached as attachment.
Thread update:
Made wiki for m3 export scripts. Registration needed at GitHub (takes only seconds[TM]). Link to project: https://github.com/InDesign/M3TOX3D Paste that link to your signature and tell a friend! Everyone are welcomed from sc2mapster forums to involve. Spread the word.
EDIT: I added to wiki all functions (both Blender and 3ds Max). There are currently only importer functions (you really do not need exporter functions because when you can read out, you can reverse that process to write exporter). Next task is to write tutorial how community can participate in this project.
So how does this help understand how to make the elusive .m3a format?
Contribute to the wiki (Wiki button at top of page) Considered easy altering of the unit textures?
https://www.sc2mapster.com/forums/resources/tutorials/179654-data-actor-events-message-texture-select-by-id
https://media.forgecdn.net/attachments/187/40/Screenshot2011-04-17_09_16_21.jpg
I lack skills to offer help to you DrSuperEvil. I read about sc2 Diablo thread. There I learned what is m3a format: a m3 file storing only animations (animated bones). In sc2 Diablo thread had some good suggestions. I understood it was agreed model_name.m3 uses animations from model_name.m3a. Rough outline would be to import into 3ds Max sc2 Diablo, delete its geometry, animate only skeleton (Diablo bones) export new file as SpaceDiablo.m3a (make link in GalaxyEditor) and look does it work. Same could be tried with simple cube model to verify process itself is valid. Create cube (triangle or plane) in 3ds Max give it one bone (so geometry would be linked to bone), save as m3. From your saved m3 file delete geometry, animate bone to rotate, save as cube.m3a. Load into GalaxyEditor and look does it work. I understand goal is to keep sc2 Diablo model as it is and do all the animaton in m3a file because (importing Diablo will lose data because nature of Blizzard close source model format, though 3ds Max import script is best what we have currently).
On thread topic: Considering EULA of SC2 and copyrights its illegal to import data out of SC2 (models). To make it short X3D2M3 is legal (somewhat) while M3TOX3D is illegal for sure. In this light my previous suggestions about Diablo model manipulating are illegal. Still our project here is good for educational, non-profit use. Its one of these things legal to own, illegal to use. If I get time I will publish tutorials how to join project, learn to help us and share learned findings on wiki page (commenting code). Additional bonus is that if you find our done work $][!", you can always make it better yourself. Just any start is always hard.
@SoodaTH: Go
Thank you very much for git repo :D just watch on pull requests ;)
So in short there is no easy way known as of yet to add new animations to existing models without altering the whole thing.
Also when did diablo come into this?
Contribute to the wiki (Wiki button at top of page) Considered easy altering of the unit textures?
https://www.sc2mapster.com/forums/resources/tutorials/179654-data-actor-events-message-texture-select-by-id
https://media.forgecdn.net/attachments/187/40/Screenshot2011-04-17_09_16_21.jpg
Diablo model, I did not know what was m3a and searched forums. Search returned Diablo model thread where I learned what is a m3a format. @Frizi I'm really beginner in GitHub. I have no idea what are all these pull requests or how they merge with my repos (though I have a slight assumption). I forgot to add in PM in GitHub to you that moving code tags between text will explode wiki-page, I have no idea why. At that time I did not know any better solution to creating environment for commenting code. Wiki had history and easy user access possibility. If you manage to make it better post here quick explanation how people can use it (access to comment code). Simplicity and quick access is good.
All I know is the civilian models and all the SM campaign cinematic stuff used .m3a as the source of their animations.
Contribute to the wiki (Wiki button at top of page) Considered easy altering of the unit textures?
https://www.sc2mapster.com/forums/resources/tutorials/179654-data-actor-events-message-texture-select-by-id
https://media.forgecdn.net/attachments/187/40/Screenshot2011-04-17_09_16_21.jpg
Who is into learning MAXScript then about structs you can read from here: http://blog.duber.cz/3ds-max/an-interesting-concept-behind-structs-in-maxscript About generic 3ds Max Maxscript (MXS) syntax: http://www.scriptspot.com/bobo/mel2mxs/mel2mxs.htm
Anyone already doing research about .m3a? It will be really good, to document it and try to write even a stub for reading/writing these files.
EDIT START Frizi how I am able to upload new files to my own repo, currently I get "note about fast-forward" error in short: https://github.com/InDesign/M3TOX3D/pull/1 How do I accept it and move on with my life?
Meanwhile I recreated in python parts of M3I_Ref struct, I learn how to read binary data (idea is to port maxcript functions which I need to Python) http://codepad.org/LY2W7OwF I added much cool reading about Maxscript to Wiki, check it out.
EDIT END
Im in the phase of learning MXS to understand better Toxicated script parts. Though I find most of script functions pretty self explanatory (I have wc3 jass background mostly, mixed with some python for humanitarian and Blender API calls). Currently puzzles me function parameters and structs like myStruct.myFunc.myMethod.myVar. In real life currently working (finished high school, went to military, came back and are going to continue studies). I understand you (like most of us) want to make this happen just like that. But I can't focus on my hobby (3d modeling and similar) fulltime. That is why I consider offering opportunity for community to participate. Maybe there are kids who don't know what to do with their free time? Maybe they could help us with this by reading our comments (made way) and make it better. I bet some think: "omg, programming!?". I once was a gamer then I read tutorials about JASS and now I do this. 3dsMax script took one year to make what it is today, SC2 took to be made how long? Our project here? Made by two days? :D I have thinked about re-creating same functions in different programming language (lets take for example python because I know it somewhat). If you understand what does function result is, you can transfer its idea to totally different programming language. Maybe some need to learn learning before they are able to learn :D Take this example (3ds Max Toxicated script): fn M3I_Main calls fnM3I_Open That is one total branch which can easily be recreated in python (I am even able to recreate it). My point is it is possible step by step to export script itself to different language. If we are eliminated all the easy things then there is greater chance that people (let it be whomever) can consentrate all their think-power to that one hard thing. By many on very specific topic it should be solved in no-time (at least faster when trying to do it alone). Dunno why people underestimate that power what we could use.
My current goal is to reproduce Toxicated mxs script way of reading tags in m3 file. My really first baby steps in mxs <> python conversion: read 33DM from Bunker.m3 file with my script (kudos to Toxicated and Muraj). What I did can be done in two ways - - - with 2 lines (or even 1 line) of code or like I did (lots of lines). By doing I learn and that is what it is all about. Codepad paste: http://codepad.org/ThZK6is4 I can only suggest others to post their progress too, then it is bit more fun and alive.
Sounds facinating. Still it is the .m3a format that interests most people so they can give their zerglings extra dance moves etc. without having to do a top down approach.
Contribute to the wiki (Wiki button at top of page) Considered easy altering of the unit textures?
https://www.sc2mapster.com/forums/resources/tutorials/179654-data-actor-events-message-texture-select-by-id
https://media.forgecdn.net/attachments/187/40/Screenshot2011-04-17_09_16_21.jpg
@DrSuperEvil: Go
No it isn't. The .m3 format is what interests the most not .m3a, even if one figured out the .m3a format you'd still need someone to make those "extra dance moves" that so many people seem to want.
Either way with Triceron we've been trying to figure out, after so many tests with blizzard's models and our own created models and m3a we've got nothing really :P Just the conclusion that the .m3 must likely have a flag or something that indicates it that it requires an .m3a or otherwise it will just ignore any attempts of .m3a you try to shove on them. They can't just be data controlled. (meaning that just making a m3a value for the "required animations" field isn't enough) An .m3 with a m3a reference always has everything bones, mesh, shaders, effects, weightning, ect but no animations. An .m3a, though doesn't it just has bones, lacks mesh, shaders and everything but has the animations. But the weightning and bones for both files have to be the same. I'm trying to compare files trying to spot a different between them (besides the ones we know) but I only know so few about programming and I'm doing this comparison by opening them in a hex editor cause obviously we don't have proper modelling tools as of yet.
EDIT 09.09.11 I decided to post about progress once in a month. Again knowledge is freely available at https://github.com/InDesign/M3TOX3D It requires people who want to do something with that knowledge. EDIT END
Was busy with real life work, going to look into m3 format in upcoming days. Importing mxs structs into python class (python way doing structs) seems pretty strait forward, that means quicker copy-paste of written functions and less manual conversion from one language to another. Hex file comparison, have not never tried it. That is how they found m3 file format in first place: by automatically parsing all m3 hex files and extracting patterns out of it. Exciting is that could we be the ones who make those modeling tools available to public.
Maybe reply is better than no reply at all. I have legally aquired 3ds max license through their student program (if you are a student in your local school then you are legit to download their software as long as you have official student status [or something like that]). Anyway I have downloaded 2x 3.3 GB of data from their site (2x 24h hours download time, my connection ~). To find out Windows 7 and 3ds Max 2010 is not compatible?! I had blue dream filling all the Toxicated script with debug messages about data manipulation which would make recreation of Toxicated scripts in Python easy. I got legit Windows 7, legit 3ds Max 2010 and they do not get along. My dirty fix to this would be to buy new HDD, install legit Windows XP on it (I can get it by school and Windows partnership) and try my luck on that set (needing new usb stick to transfer my precious software [3ds]. My time to summer start is planned and there isn't much room for script development. To my suprise there is EASY way to download and install LEGALY Autodesk 3d Max on your pc IF you are a student (in ANY school around the world), currently registering and starting 3ds max download is as easy as is making a new email address. Additionally 3ds Max can be registered as a 30 days trial (if you are desperate trial install + OS format after trial should give you hint) I write once more the idea I had. Maybe some bright mind does it before me: 1. Download Autodesk 3ds Max 2010 (install as trial) 2. Stack Toxicated code full of debug messages to see how script works on test m3 file (bunker.m3 would be good) 3. Dump all [variable data] gotten data to file and upload for others to see 4. Rebuild similar script in free to use environment (I thoguht Python + Blender 3d program).
I'm using Max 2010 with W7, I don't think it's an issue of incompatability.
Then it might be issue with my odd computer, lately it acts strangely.