Carl Worth [Thu, 27 Jun 2013 03:46:40 +0000 (20:46 -0700)]
configure: Set GL_LDFLAGS and EGL_LDFLAGS in configure script
The libfips library doesn't link directly to libGL nor libEGL so
didn't need these flags. But we're adding test programs that do link
to these, so the test's Makefile needs access to these flags.
Carl Worth [Thu, 27 Jun 2013 01:18:52 +0000 (18:18 -0700)]
Push final collection of CFLAGS/LDFLAGS from Makefile.config to Makefile.local
This makes the final decision more explicit closer to where the flags
are actually being used. This will be helpful as we add other
programs, which can now easily mimic the style of flags collection as
is done for fips.
This also eliminates any potential confusion of FIPS_FLAGS
vs. FINAL_FIPS_FLAGS, etc. The use of "FINAL_" has now been entirely
eliminated.
Carl Worth [Mon, 24 Jun 2013 22:49:44 +0000 (15:49 -0700)]
eglwrap: Add comment describing why we don't lookup into libGLESv2.so
A user recently asked me why we didn't perform lookups in
libGLESv2.so, (instead of just libEGL.so). I actually made the mistake
of writing code to do that before I realized the answer.
Adding the answer in a comment here should help me avoid making that
mistake again.
Carl Worth [Mon, 24 Jun 2013 22:44:47 +0000 (15:44 -0700)]
EGL: Add wrapper for eglGetProcAddress
If an EGL-using program uses eglGetProcAddress to locate functions, we
want to intercept that to return our own versions of the functions,
(to add out metrics timings, etc.).
If the requested function is not implemented in our library, just
defer to the real, underlying eglGetProcAddress function to find the
symbol.
Carl Worth [Mon, 24 Jun 2013 22:27:50 +0000 (15:27 -0700)]
configure: Fix configure check to look for egl.h in the correct directory
This configure check was broken by looking for GL/egl.h instead of
EGL/egl.h as it should. This failure was masked on any system with an
EGL implementation providing a pkg-config file (egl.pc).
Carl Worth [Mon, 24 Jun 2013 20:22:59 +0000 (13:22 -0700)]
fips-dispatch: Completely separate fips-dispatch GL prototypes from GL/gl.h
Move the OpenGL prototypes previously in fips-dispatch.h to a new
fips-dispatch-gl.h. The idea here is that any given file should
include only one of GL/gl.h or fips-dispatch-gl.h.
Files that implement libfips wrappers for OpenGL functions include
GL/gl.h to ensure that they implement functions with the correct
prototypes.
Meanwhile, files that call into OpenGL functions at run-time, (such as
metrics.c which calls the various OpenGL query-related functions),
instead include fips-dispatch-gl.h and do not include GL/gl.h. With
this approach, any new calls to OpenGL functions will cause
compilation warnings if the stubs are not also added to
fips-dispatch-gl.h.
Carl Worth [Sat, 22 Jun 2013 00:10:03 +0000 (17:10 -0700)]
Add dynamic dispatch for any calls to OpenGL functions.
Previously, fips code was making direct calls to OpenGL functions,
(such as glGenQueries, glBeinQuery, etc. within metrics.c). Some
OpenGL implementations do not export these symbols directly,
(expecting the application to instead look the symbols up via a call
to glXGetProcAddressARB or eglGetProcAddress).
The new fips-dispatch code added here does precisely that, (and adds
wrapper for both glXMakeCurrent and eglMakeCurrent in order to know
which GetProcAddress function should be called).
The dispatch code follows the model of piglit-dispatch, (available
under the same license as fips). Thanks to Eric Anholt for suggesting
following the same approach as piglit.
Carl Worth [Mon, 24 Jun 2013 20:19:41 +0000 (13:19 -0700)]
configure: Fix to have compiler warnings enabled while building libfips
In commit e42d9f224a4ef2784f8fd43f9f4f5c593a7ddd57 , when the flags
were split between fips and libfips, the warnings flags were
mistakenly applied to both CFLAGS and LDFLAGS of fips. (What was
actually intended was to have the warnings applied to the CFLAGS of
both fips and libfips).
Carl Worth [Fri, 14 Jun 2013 18:29:33 +0000 (11:29 -0700)]
TODO: Add some additional items suggested by Eero.
Again, simply trying to ensure that good ideas that come in via email
don't get dropped on the floor.
Report shader compilation time.
+Report elapsed time per frame.
+
+Add options to control which metrics should be collected.
+
Add Eric's tiny hash table for collecting per-shader statistics
Carl Worth [Fri, 14 Jun 2013 18:11:42 +0000 (11:11 -0700)]
TODO: Remove bug causing libfips-32.so to not build.
The configure script now checks for this problem and won't attempt to
build a non-native-arch libfips if the toolchain pieces, (such as the
gcc-multilib package), are not in place.
Carl Worth [Fri, 14 Jun 2013 06:29:07 +0000 (23:29 -0700)]
configure: Fully separate CFLAGS/LDFLAGS between fips and libfips
The top-level program and the underlying library have fundamentally
different requirements. For example, the top-level program needs to
link against libtalloc and libelf but the library does not.
Previously, the necessary flags for both were mixed together in
CONFIGURE_CFLAGS and CONFIGURE_LDFLAGS. This caused the library to
unnecessarily link against libtalloc and libelf, (which in turn caused
problems since the library is compiled as both 32-bit and 64-bit but
the system may not provide both 32- and 64-bit versions of these
libraries).
By splitting things up into separate FIPS_LDFLAGS vs. LIBFIPS_LDFLAGS,
etc. we can keep the dependencies down to what is really required and
eliminate several sprious failure cases.
Carl Worth [Fri, 14 Jun 2013 05:02:45 +0000 (22:02 -0700)]
Add more detailed warning if libfips binary could not be found.
Since we now may not compile a non-native bit-size libfips, (such as
not compiling libfips-32.so on a native 64-bit install), then fips can
fail when trying to wrap a 32-bit application.
So be kind and suggest that the user install gcc-multilib and
re-compile fips as one potential solution for this problem.
Carl Worth [Thu, 13 Jun 2013 22:39:30 +0000 (15:39 -0700)]
configure: Move the code which detects compiler warning options
For no good reason, this code was previously stuck right in the middle
of code reporting the final results of previous checks, (after the
case reporting that errors were found and before the case reporting
that no errors were found).
It's more clean to have the warning-option detection happen before any
of that reporting.
Carl Worth [Thu, 13 Jun 2013 22:00:49 +0000 (15:00 -0700)]
configure: Add an early check for a functional C compiler
Without this, later compiler-based checks for header files, etc. would
falsely claim that libraries were not installed, (when in fact, the
libraries were installed but the compiler-based checks were failing
because no compiler is available).
Carl Worth [Thu, 13 Jun 2013 21:26:08 +0000 (14:26 -0700)]
configure: Move the error message for pkg-config earlier
Once pkg-config cannot be found, there's no point in running the rest
of the configure script since it will just spew a bunch of errors due
to missing pkg-config.
So, stop early and let the user know that pkg-config needs to be
installed.
Carl Worth [Wed, 12 Jun 2013 23:22:06 +0000 (16:22 -0700)]
Fix to actually load the real libGL.so when the application asks for it.
Our wrapper library intercepts calls to dlopen "libGL.so" and returns
a handle to itself. That's correct, and as intended.
Before doing this, however, it's essential to actual dlopen "libGL.so"
for real even though we won't be returning a handle to it. This
ensures that any side effects from that dlopen are taken care of.
This fixes a bug where "fips apitrace replay foo.trace" would cause
Mesa to fail to load its driver as follows (with LIBGL_DEBUG=verbose):
Carl Worth [Wed, 12 Jun 2013 00:18:12 +0000 (17:18 -0700)]
Switch from glGetQueryObjectiv to glGetQueryObjectuiv
The latter is available in OpenGL ES 3 while the former is not.
We don't really care about the signedness either way, (we're fetching
a Boolean), so sticking consistently to the unsigned version provides
better portability with no downside.
Carl Worth [Mon, 10 Jun 2013 21:40:43 +0000 (14:40 -0700)]
Add a new GLWRAP_DEFER_WITH_RETURN macro.
This allows for the elimination of some code duplication from our
implementation of glXGetPrcAddressARB. The previous implementation
duplicated code from glwrap_lookup simply because the GLWRAP_DEFER
macro did not provide access to the return value of the wrapped
function.
With the new macro, (very much like GLWRAP_DEFER but accepting a
parameter for a variable to accept the return value), we can eliminate
this code duplication.
Of course, our symbol-extraction script is now a bit more complicated
since it has to find occurrences of DEFER_WITH_RETURN in addition to
occurrences of DEFER, and pull out the function name as the second
argument rather than the first.
Carl Worth [Mon, 10 Jun 2013 21:34:26 +0000 (14:34 -0700)]
Remove typedef for fips_glXGetProcAddressARB_t
There were two problems with this typedef. First, we don't actually
need it, (we define it here and then use it exactly once on the next
line---it's simpler to have the direct syntax for a function returning
a pointer to a function accepting void and returning void.
More importantly, the typedef was relying on the type __GLXextFuncPtr
being defined. This happens to work with Mesa on my system but is
inherently fragile. So the code is more robust not relying on this.
Carl Worth [Thu, 30 May 2013 22:10:15 +0000 (15:10 -0700)]
Makefile: Automatically generate libfips.sym symbol map
Previously, we had to manually maintain this table of symbols, (hence
it was too easy for the list to be stale). Instead, we now generate
the list automatically by examining the source code for symbols that
are wrapped.
These were previously commented out with a "FIXME" comment. I don't
recall now what original problem I ran into when trying to add these,
but they seem to work fine now.
Carl Worth [Thu, 30 May 2013 21:05:08 +0000 (14:05 -0700)]
metrics: Collect new timer-query results before report
The timers from the most-recently-completed frame may not yet be
complete, but still, for the most accurate reports, we now collect all
available timer values before printing the report.
Carl Worth [Thu, 30 May 2013 19:55:26 +0000 (12:55 -0700)]
dlwrap: Fix dlwrap_real_dlopen to only perform dlsym-lookup once
Making the real_dlopen variable static means that we perform the dlysm
lookup only once and then re-use the result for later calls. This is
what was always intended for this code.
Carl Worth [Fri, 24 May 2013 18:11:15 +0000 (11:11 -0700)]
Avoid inserting timer queries while constructing a display list
We only want to time actual drawing operations. When between glNewList
and glEndList, calls that look like drawing operations are not really,
instead these are just calls that are being recorded to be later
executed with glCallList. (And it won't work to put our timer queries
inside the display list.)
So, track when we are within glNewList/glEndList and don't add timer
queries. Instead, we will time these operations as a whole with a
timer query around the glCallList call itself.
Carl Worth [Thu, 23 May 2013 21:22:42 +0000 (14:22 -0700)]
Add several missing symbols to the libfips symbol map.
I'm not sure why these weren't added before, but without these in the
map, these calls were not being successfully wrapped, so these calls
were not being timed as they should have been.
In the future we should automatically generate the symbol list to
avoid any similar problems.
Carl Worth [Mon, 6 May 2013 18:57:24 +0000 (11:57 -0700)]
Remove useless fork before executing wrapped program.
The fork did nothing for us since the parent simply waited on the child
and then exited. It's simpler to simply exec the wrapped program, (which
simplifies running fips within a debugger, etc.).
Carl Worth [Mon, 6 May 2013 18:18:33 +0000 (11:18 -0700)]
Move metrics-tracking code from glwrap.c to new metrics.c
We're moving toward having separate *wrap.c files for each flavor of GL,
(glwrap.c, glxwrap.c, and eglwrap.c). But this metrics-tracking code is
generic to all of those so belongs in a separate module.
Carl Worth [Thu, 2 May 2013 21:33:12 +0000 (14:33 -0700)]
Reduce code duplication with with new 'glwrap_lookup' function.
We already had two copies of 'lookup', (and were anticipating additional
modules which needed it as well). The DEFER macro is now also exported
as GLWRAP_DEFER for use in additional modules.
Carl Worth [Mon, 29 Apr 2013 21:54:52 +0000 (14:54 -0700)]
configure: Add checks for GL/gl.h
Previously, the compile would just forge ahead assuming GL/gl.h was present.
Now, at configure time, actually look for gl.h, (first, by looking for
a pkg-config "gl" package, otherwise, trying to just test-compile
something with a #include <GL/gl.h>).
If things aren't found at configure time, tell the user which packages
to install.
Carl Worth [Mon, 29 Apr 2013 20:23:58 +0000 (13:23 -0700)]
Fix glwrap.c to workaround 'const' changes in OpenGL headers.
Not all OpenGL headers are created equal. Some include more "const"
keywords than others, and we don't know wheter the headers we are
compiling against include the extra "const". We force all to be equal
by using the preprocessor to remove all "const" keywords altogether.
And in each wrapper we fire of a glBeginQuery/glEndQuery measurement around
the call to measure how much GPU time is consumed by the call. At the end
of each frame, we capture all available query results and accumulate those
into per-shader-program counters based on the currently active program,
(which we track by wrapping glUseProgram and glUseProgramObjectARB).
Finally, every 60 frames, we print out a simple report showing the
accumulated time for each shader program.
The report could very easily become more sophisticated. Here are some
obvious ideas:
1. Sort the report so that the most active shaders are reported first
2. Come up with some real units for the report values rather than mega-ticks
3. Report relative execution time as percentages
4. Clear the screen for each report, (with ncurses)
5. Dump the source for the shaders themselves to a file for easy inspection
Without any of the above, things are fairly raw for now, but are
perhaps still useful.
Carl Worth [Thu, 25 Apr 2013 05:55:01 +0000 (22:55 -0700)]
Generalize glXGetProcAddressARB wrapper to work for all wrapper functions
The originally implementation here had a whitelist of function names for
which we would return a wrapped symbol, (a very short whitelist consisting
only of "glXSwapBuffers"). A hard-coded list here would be a maintenance
nightmare.
Instead, we now simply perform a dlsym lookup on the wrapper library itself
and if there's a function that exists in the library matching the name
being requested, we return that.
This way we can add functions to our wrapper library without needing to
change the implementation of glXGetProcAddressARB at all.
Carl Worth [Wed, 24 Apr 2013 22:28:43 +0000 (15:28 -0700)]
Add a simple fips.h file.
This pulls in a few widely-used header files, (config.h, stdio.h,
stdlib.h, and string.h), and also gives us a place to define common
macros such as unused and STRNCMP_LITERAL.
Carl Worth [Wed, 24 Apr 2013 08:08:54 +0000 (01:08 -0700)]
Append to, rather than replace, the LD_PRELOAD value.
This is more polite for running things such as Steam games where there
is already an LD_PRELOAD value in place. This way, both the Steam overlay
and fips can get along happily.
Carl Worth [Wed, 24 Apr 2013 07:58:26 +0000 (00:58 -0700)]
Add wrappers for dlopen, dlsym, and glXGetProcAddressARB
This allows for many applications to start working with fips that would not
work before. Specifically, applications that dlopen libGL.so.1 instead of
directly linking with it would previously bypass fips' attempts to wrap
GL calls.
With these new wrappers carefully in place, many applications now work.
I've verified the following applications at least:
apitrace replay
NightSkyHD, a Humble Bundle game, both 32 and 64-bit versions
World of Goo, via Steam
I was also happy to notice that the Steam overlay does not cause fips
any difficulties.
Carl Worth [Wed, 24 Apr 2013 00:47:20 +0000 (17:47 -0700)]
Fix fips to work without requiring an absolute path for program to run.
With the latest commit that examines the ELF header of the program to
run fips was suddenly requiring that the absolute path of the program
be provided. This was obvisouly not desired.
It's simple enough to search through the PATH environment variable to
find the absolute path of the program to be run and examine that.
Carl Worth [Wed, 24 Apr 2013 00:25:13 +0000 (17:25 -0700)]
Compile both 32-bit and 64-bit versions of the wrapper library.
This is intended to make it transparent to run fips with either a
32-bit or a 64-bit program. And it does do that once you successfully
build both versions of the library.
Actually being able to build both versions of the library is a little
tricky though. Here are some of the tricks:
1. You will need to have installed both a 32-bit and a 64-bit .so file
for each dependent library, (currently libelf and libtalloc).
The current configure script doesn't check for both versions, so
you don't get a lot of guidance here. And that's because...
2. On Debian, at least, one cannot currently install both
libtalloc-dev:amd64 and libtalloc-dev:i386 at the same time.
Contrast with libelf-dev:i386 and libelf-dev:amd64 which work just
fine when installed simultaneously.
One can work around this by just install libtalloc-dev:amd64 and then
manually creating the link you need for the i386 package. Namely:
Carl Worth [Tue, 23 Apr 2013 21:17:33 +0000 (14:17 -0700)]
Start wrapping OpenGL, and print periodic FPS value to stdout.
In addition to the fips binary, we now also compile a libfips.so library
and LD_PRELOAD that before executing the program specified on the command-
line.
The libfips.so library wraps OpenGL calls of interest for purpose of
instrumentation. So far, the only call wrapped is glXSwapBuffers and
the only instrumentation is to compute and print out a frames-per-second
value every 60 frames.