Horde3D

Next-Generation Graphics Engine
It is currently 16.04.2024, 07:21

All times are UTC + 1 hour




Post new topic Reply to topic  [ 9 posts ] 
Author Message
PostPosted: 31.01.2011, 11:04 
Offline

Joined: 30.01.2011, 03:33
Posts: 5
What I'd also like to see is platform independent utility functions to do the following:
* Get resource paths relative to each platform's default resource path.
* Save and load preferences in an operating system friendly way.

I do realise it's probably in the domain of GLFW or an equivalent API to do this, but if you have some resource utility functions why not have some more. Different operating systems have different conventions with file layout and it seems a lot of people on these forums myself included have been having trouble with resources not being found when running the demos.

On MacOSX it is recommended to use the API to find a resource's path given a filename, filetype, and optionally subdirectory. These resources are then found in a resource directory in a self contained app bundle. I think Horde3D could profit from having mechanisms to find resources like this, but in a platform independent way, like this:
Code:
lightPath = h3dResourcesGetPath("light.material", "xml", "materials/lights");


Additionally most operating systems have a means to store things like numbers, strings, and other settings in some form of associative array in a centrally managed location. This is User Defaults on OSX, probably in the registry on windows, and perhaps ~/.mysettingshere on Linux, it'd be nice to be able to save or load settings like this:
Code:
h3dSettingsSaveInt32("window.width", 800);
h3dSettingsStore();


I intend to do something like this, let me know if anyone wants to help, or if you think it will be useful to you.

This code is not what I'm describing, but related and might be useful to someone, it has been ripped from the demos and tries to be easy to integrate into them, it finds the resource path in an MacOSX app's bundle.
Code:
#ifdef __APPLE__
#include <CoreFoundation/CoreFoundation.h>
#endif

std::string getDefaultBaseResourcesPath(int argc, const char * argv[])
{
   if (argc == 0)
      return NULL;

#ifdef __APPLE__
   CFBundleRef bundle;
   CFURLRef resourceDir;
   CFURLRef absoluteResourceDir;
   CFStringRef systemPath;
   char * buffer;
   size_t buffer_size;

   // extract the resource path from the main bundle
   bundle = CFBundleGetMainBundle();
   resourceDir = CFBundleCopyResourcesDirectoryURL( bundle );
   absoluteResourceDir = CFURLCopyAbsoluteURL( resourceDir );
   systemPath = CFURLCopyFileSystemPath( absoluteResourceDir, kCFURLWindowsPathStyle );

   // turn the resource path into a c string on the stack
   buffer_size = CFStringGetMaximumSizeForEncoding(CFStringGetLength(systemPath),
                                       kCFStringEncodingUTF8);
    buffer  = (char*) alloca(buffer_size + 1);
    CFStringGetCString(systemPath, buffer, buffer_size+1,
                  kCFStringEncodingUTF8);

   // release temporary resources
   CFRelease(systemPath);
   CFRelease(absoluteResourceDir);
   CFRelease(resourceDir);

   return std::string(buffer) + "\\";
#else
   const std::string s( argv[0] );
   if( s.find( "/" ) != std::string::npos )
      return s.substr( 0, s.rfind( "/" ) ) + "/";
   else if( s.find( "\\" ) != std::string::npos )
      return s.substr( 0, s.rfind( "\\" ) ) + "\\";
   else
      return "";
#endif
}


Top
 Profile  
Reply with quote  
PostPosted: 01.02.2011, 12:49 
Offline
Tool Developer

Joined: 13.11.2007, 11:07
Posts: 1150
Location: Germany
As you already mentioned too, I think thats not the domain of Horde3D. You can load resources from where ever you want, and there is no "default" resource directory within Horde. Maybe in the samples, but then we would have to add some platform independent resource functions to the samples and not to Horde3D.


Top
 Profile  
Reply with quote  
PostPosted: 02.02.2011, 07:32 
Offline

Joined: 08.11.2006, 03:10
Posts: 384
Location: Australia
Sounds like it could be useful for some people, but it definitely doesn't belong in the core H3D library (which is completely disconnected from "files" or the file-system), it should go in the Utils library (which does contain file-relation functions).


Top
 Profile  
Reply with quote  
PostPosted: 02.02.2011, 09:06 
Offline
Tool Developer

Joined: 13.11.2007, 11:07
Posts: 1150
Location: Germany
But still the question is what should be the "default" resource directory, and how to find it? E.g. under Windows Mobile there isn't even a working folder (ok, currently we don't have a running windows mobile implementation and windows mobile probably won't have a big future :-) ). And considering the preferences, I think there are things like QSettings if you're using Qt. For an application that uses Horde, you always have to add other components (like GLFW for the very basic window handling, etc.). So I think it shouldn't to much of a problem to include other code for that.

But if someone has the time to write a compact clean cross-platform solution for the things mentioned by therealbnut we can think about it again. For myself I currently don't really see the requirements for something like that.


Top
 Profile  
Reply with quote  
PostPosted: 03.02.2011, 00:38 
Offline

Joined: 30.01.2011, 03:33
Posts: 5
I agree that there isn't a default resource directory on many platforms, but some platforms like MacOSX have such a directory. If there wasn't a platform default directory, or it is unspecified then it would most likely default to the working directory of the executable. Some platforms, notably embedded devices, might not even allow for direct filesystem access.

Volker wrote:
But if someone has the time to write a compact clean cross-platform solution for the things mentioned by therealbnut we can think about it again. For myself I currently don't really see the requirements for something like that.

This is about more flexible, unambiguous and concise expression of information. I will do what I can then post it here, and I understand it will probably make little difference to a lot of users, specifically Windows ones. But I think it will make Horde3D a lot more flexible and future proof. I'm just trying to remove barriers to entry for other platforms, both as a developer starting out with Horde3D and a developer trying to deploy.

As for this not being appropriate for the core Horde3D library, I have agreed from the start, but I didn't think it through and wrote h3d instead of h3du on my examples. My intention was to supplement or even replace h3dutLoadResourcesFromDisk. However currently h3dAddResource which is not a utility function references files with an inflexible file path that is open to a lot of platform specific interpretation.

Before I get crucified can I just say it's the design first, not the features of Horde3D that brought me here. You guys have done an awesome job and I'm just trying to give back in that same spirit.

After rethinking it and analysing the internal workings of Horde3D and the Horde3D Utility library I might take the controversial stance that h3dAddResource should be changed, or supplemented. My reasons are as follows:
  • h3dGetResName is treated as a path, not a name.
  • Paths even as simple as ./foo/bar.bas make numerous assumptions of the underlying filesystem.
  • Different platforms want different resource identifier strings (ie. file path), which requires consistent input or robust interpretation.

Feel free to criticise and scrutinise this, design is ultimately judged by those that use it:

Proposed system
Code:
H3DRes h3dAddResource( int type, const char *context, const char *name, int flags );
H3DRes res = h3dAddResource( H3DResTypes::Pipeline, "default/pipelines", "forward.pipeline", 0 );
h3dutSetResourceWorkingDirectory("./resources/|./other_resources/");
h3dutLoadResources();


New functions (in the context of the example code)
  • h3dGetResName(res) returns "forward.pipeline"
  • h3dGetResContext(res) returns "default.pipelines"
  • h3dutLoadResources() works like h3dutLoadResourcesFromDisk does now, but internally it calls h3dGetResData.
  • h3dGetResData(res, &data, &data_size) returns true we hope :P. Loading the resources in the way most suited to each platform, from a bundle, disk, or even a resource in the executable. It will derive the file extension if necessary from h3dGetResType. The data and size of that data are stored in the variables pointed to by its two arguments.
  • h3dutSetResourceWorkingDirectory provides the functionality of the arguments in h3dutLoadResourcesFromDisk if needed, but might be ignored on some platforms.


Top
 Profile  
Reply with quote  
PostPosted: 03.02.2011, 04:28 
Offline

Joined: 08.11.2006, 03:10
Posts: 384
Location: Australia
therealbnut wrote:
I understand it will probably make little difference to a lot of users, specifically Windows ones. But I think it will make Horde3D a lot more flexible and future proof. I'm just trying to remove barriers to entry for other platforms, both as a developer starting out with Horde3D and a developer trying to deploy. ... My intention was to supplement or even replace h3dutLoadResourcesFromDisk.
It makes little difference to me because I don't use H3DUT whatsoever ;)
But yeah, for people who do want all the utilities to get H3D up and running quickly, then better file-system support is obviously of use.
Quote:
However currently h3dAddResource which is not a utility function references files with an inflexible file path that is open to a lot of platform specific interpretation.
Actually h3dAddResource deals with resource names only. You can choose to use a file-path as a name if you want to, but the function does not have to be used that way (and IMHO OS-file-paths should *not* be passed to that function).
Quote:
h3dGetResName is treated as a path, not a name.
Whether or not this is true is up to the application. Horde itself (h3d core) does not treat them as paths, only as names.
It's up to the application to choose some kind of mapping between named-resources and how to obtain the data for those resources. The current utility library has made the choice of simply using the name as a path, which is causing the problems you're trying to address. To reiterate, this problem of treating names as paths, lies within the utility library, not the core.


Last edited by DarkAngel on 03.02.2011, 04:43, edited 2 times in total.

Top
 Profile  
Reply with quote  
PostPosted: 03.02.2011, 04:39 
Offline

Joined: 30.01.2011, 03:33
Posts: 5
I'll refrain from quoting your entire message DarkAngel, thanks :). There are a few things there that have cleared things up for me. As Volker said it's just a matter of someone making it, and then seeing if the community takes it up.


Top
 Profile  
Reply with quote  
PostPosted: 03.02.2011, 04:43 
Offline

Joined: 08.11.2006, 03:10
Posts: 384
Location: Australia
As an example of an alternative way to use h3dAddResource -- I'm of the opinion that games don't even need hierarchical file systems. Every asset in my project has a unique name, but no path.

In development builds, there is a data folder, containing game assets in whatever folder hierarchy the developers are comfortable with. A separate program that lives in the system tray on their development PC maintains an index of all the files in this directory hierarchy. When a game-system (such as horde, but including other libraries as well) requests a named asset, this name is sent over TCP/IP to the file-serving application, which loads the file and sends the data back to the game. The game and file-server don't have to run on the same machine, but usually do during Windows development.
N.B. if there are two files with the same name in different parts of this folder hierarchy, the file-server app will issue a warning to the developers to clean up their mess.

In shipping builds, all of the files from the data folder are packaged up into one big archive. When the game needs to load a named resource, it looks up that name in the archive's index and reads the correct sub-section of the archive file.


Top
 Profile  
Reply with quote  
PostPosted: 03.02.2011, 06:24 
Offline

Joined: 02.02.2011, 19:14
Posts: 5
DarkAngel wrote:
As an example of an alternative way to use h3dAddResource -- I'm of the opinion that games don't even need hierarchical file systems. Every asset in my project has a unique name, but no path.


That's a neat idea. I'll adopt that too. Managing materials in model subfolders is really confusing if you want to reuse them.

DarkAngel wrote:
In shipping builds, all of the files from the data folder are packaged up into one big archive. When the game needs to load a named resource, it looks up that name in the archive's index and reads the correct sub-section of the archive file.

What do you use for archiving data? I'm planning to add something like that too, but wouldn't know which of the many archive formats to use for a task like this.


Top
 Profile  
Reply with quote  
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 9 posts ] 

All times are UTC + 1 hour


Who is online

Users browsing this forum: No registered users and 12 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group