Horde3D
http://horde3d.org/forums/

Case sensitive resources
http://horde3d.org/forums/viewtopic.php?f=3&t=390		 Page 1 of 1
Author:  blue [ 01.07.2008, 18:00 ]
Post subject:  Case sensitive resources
Currently Horde is case sensitive for resources. This is generally a good idea for an Engine that runs on Unix based systems, but it leads to some non-obvious results from user error. Since shaders, materials and textures are specified in the xml’s then it is relatively easy to spell them with different cases. Under windows this won’t cause an error, but will instead duplicate the resource.

So for instance if I write “parrallax.shader.xml” in one material and “Parrallax.shader.xml” in another, two shader resources are created and two shaders are uploaded. Doing this has a negative effect on load times and performance. Although it is caused by user error, it’s a non-obvious result and can easily go undetected. Similar problems can exist with texture resources, where texture data will be duplicated I believe.

Should Horde3D be case insensitive on case insensitive platforms?
Author:  swiftcoder [ 01.07.2008, 19:11 ]
Post subject:  Re: Case sensitive resources
blue wrote:
Should Horde3D be case insensitive on case insensitive platforms?
As a general rule, cross-platform software should work exactly the same on all platforms (though this isn't always possible). Introducing Windows-specific behaviour that will break when ported to mac or Linux is a bad idea™.
Author:  marciano [ 01.07.2008, 23:20 ]
Post subject:  Re: Case sensitive resources
Yeah that's something I'm also not sure about. At first resource names were case-insensitive but to make everything consistent and add better support for the non-Windows platforms I made also the resource names (which are not necessarily file names, you make the mapping yourself) case sensitive. But I see the problem you describe. Maybe we could add an engine option that makes the resource manager convert all names to lower case, so you can choose on a per-project basis what you want.
Author:  DarkAngel [ 02.07.2008, 04:55 ]
Post subject:  Re: Case sensitive resources
swiftcoder wrote:
blue wrote:
Should Horde3D be case insensitive on case insensitive platforms?
As a general rule, cross-platform software should work exactly the same on all platforms (though this isn't always possible). Introducing Windows-specific behaviour that will break when ported to mac or Linux is a bad idea™.
I see your point, but the way it is at the moment, the engine is using Mac/Linux-specific behaviour that will break when ported to Windows. So we're damned either way!
Author:  swiftcoder [ 02.07.2008, 05:55 ]
Post subject:  Re: Case sensitive resources
DarkAngel wrote:
I see your point, but the way it is at the moment, the engine is using Mac/Linux-specific behaviour that will break when ported to Windows. So we're damned either way!
Not really. The existing behaviour works fine on all platforms - unless you have multiple assets only differentiated by case, which would be plain stupid. The samples all find their assets on Windows or Linux, with equal ease.

The only thing that case-insensitivity will do is cover programmer errors (requesting the same file multiple times, in a different case each time). These are errors in C++ (in pretty much all other languages, and intentionally so), and I don't see much reason to make the engine more lenient than its host language in this respect.

There is also the issue of implementation - the engine is already case-insensitive with respect to files (as all Windows programs are automatically), but to make resources case-insensitive is a major hassle. It requires providing custom comparison functions to the std::containers in use (mainly std::map), and introduces significant overhead - this is the reason that case-insensitivity was scrapped during the design of python, as nobody wanted the performance hit.

Lastly, this isn't only an issue that affects Windows. Both Mac and Linux can (and often do) use case-insensitive disk formats, which means that we have to behave predictably on those as well.
Author:  kal [ 02.07.2008, 08:22 ]
Post subject:  Re: Case sensitive resources
you know the platform at compile time. so platform compatibility and implementation are no problem. (you can use different functions for win and nix, so overhead will be little).

But it is unnecessary IMHO.(it would be fun though: figuring out that nix is case sensitive, after porting your windows app. and hours of debugging ... priceless :P )
Author:  DarkAngel [ 02.07.2008, 09:02 ]
Post subject:  Re: Case sensitive resources
swiftcoder wrote:
There is also the issue of implementation - the engine is already case-insensitive with respect to files (as all Windows programs are automatically), but to make resources case-insensitive is a major hassle. It requires providing custom comparison functions to the std::containers in use (mainly std::map), and introduces significant overhead - this is the reason that case-insensitivity was scrapped during the design of python, as nobody wanted the performance hit.
Or you just convert all path-strings to lower-case as they are entered into the system.
Quote:
[if] you have multiple assets only differentiated by case, [then that] would be plain stupid.
Which means it should be safe to force all file-names into lower-case ;P

I think a fair solution would be to do this (convert all paths to a standard case-ness) when compiling for any platform which is case-insensitive (i.e. Windows). This fixes blue's issue of loading the same resource twice and shouldn't impact any other platforms.
Author:  kal [ 02.07.2008, 09:16 ]
Post subject:  Re: Case sensitive resources
wouldnt md5 sum and size of the file be better than name of the file, for uniqueness test.

so you can load bleh.jpg and blah.jpg
but only bleh.jpg is loaded because blah.jpg has the same size and md5 sum

looking this way, it is resource management related
not just case sensitivity
Author:  swiftcoder [ 02.07.2008, 11:29 ]
Post subject:  Re: Case sensitive resources
DarkAngel wrote:
swiftcoder wrote:
[if] you have multiple assets only differentiated by case, [then that] would be plain stupid.
Which means it should be safe to force all file-names into lower-case ;P
Except that you are forcing the user into a certain naming convention. I happen to like naming my files in CamelCase, but on case-sensitive hard drives, I would be forced to use all_lower_case.

Quote:
I think a fair solution would be to do this (convert all paths to a standard case-ness) when compiling for any platform which is case-insensitive (i.e. Windows).
How do we define a platform that is case-insensetive? Macs are case insensitive if you format the hard drive with that option checked. Linux is case insensitive if you are using an NTFS or FAT32 partition (both of which can happen on Windows-like distros).
Author:  DarkAngel [ 03.07.2008, 02:59 ]
Post subject:  Re: Case sensitive resources
kal wrote:
wouldnt md5 sum and size of the file be better than name of the file, for uniqueness test.

so you can load bleh.jpg and blah.jpg
but only bleh.jpg is loaded because blah.jpg has the same size and md5 sum

looking this way, it is resource management related
not just case sensitivity
That's an interesting way to look at it - using a md5/hash just makes the file-name problem disappear :D
However, md5 sums are not cheap - loading times would be much longer if you had to sum the files as you load them (unless you pre-calculate all the MD5's and save them alongside the files, in which case it would be hard to keep them up-to-date)

swiftcoder wrote:
happen to like naming my files in CamelCase, but on case-sensitive hard drives, I would be forced to use all_lower_case.
Yeah, same. Hence the ";P" ;)

Quote:
How do we define a platform that is case-insensetive? Macs are case insensitive if you format the hard drive with that option checked. Linux is case insensitive if you are using an NTFS or FAT32 partition.
Darn... Lets say there's 3 categories: (A)=always case-insensitive, (B)=always case-sensitive, (C)=can be either.

Forcing lower-case paths on-input would make (A) error-free... but obviously breaks (B), and often breaks (C).

Allowing any-case paths is not a problem on (B)... but can cause the obscure resource-duplication bug on (A) and potentially on (C).

It doesn't look like there is a solution.
But as you said - "The existing behaviour works fine on all platforms - unless you have multiple assets only differentiated by case, which would be plain stupid." - so maybe we should just note this "bug" down as "undefined behaviour", and have the engine be able to issue you with a warning if you ever load two resource files that only differ by upper/lower case.
Author:  kal [ 03.07.2008, 08:14 ]
Post subject:  Re: Case sensitive resources
can be something cheaper, like crc32 but it is not as safe as md5.
but still pointless. one could just remove the duplicates at release
Page 1 of 1 All times are UTC + 1 hour
Powered by phpBB® Forum Software © phpBB Group
https://www.phpbb.com/