Mac OS X Programming

This is a page that covers lower-level or general Mac OS X programming topics, i.e. topics not necessarily related to Cocoa, iOS and other high-level stuff.

Library related tools

otool

Displays specified parts of mach-o binaries (object files, shared libraries, executables). Probably most frequently used for displaying the names and version numbers of the shared libraries and frameworks that the object file uses:

nargothrond:~ --> otool -L $(which otool)
/usr/bin/otool:
	/usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 125.2.0)

Another use is to display the load commands in a mach-o binary. The information includes

• The order in which things get loaded when the program is started
• Which shared libraries are loaded (LC_LOAD_DYLIB and LC_LOAD_WEAK_DYLIB commands)
• Whether an entire library is weakly linked (the LC_LOAD_WEAK_DYLIB command); see the section Weak linking
• Any run paths embedded in the binary (LC_RPATH commands); see the section Dynamic library loading

nargothrond:~ --> otool -L $(which otool)
/usr/bin/otool:
Load command 0
      cmd LC_SEGMENT_64
  cmdsize 72
  segname __PAGEZERO
[...]
Load command 1
      cmd LC_SEGMENT_64
  cmdsize 632
  segname __TEXT
[...]
Section
  sectname __text
   segname __TEXT
[...]
Section
  sectname __symbol_stub1
   segname __TEXT
[...]
more sections from the __TEXT segment
[...]
Load command 2
      cmd LC_SEGMENT_64
  cmdsize 632
  segname __DATA
Section
  sectname __program_vars
   segname __DATA
[...]
more sections from the __DATA segment
[...]
Load command 3
      cmd LC_SEGMENT_64
  cmdsize 72
  segname __LINKEDIT
[...]
Load command 4
            cmd LC_DYLD_INFO_ONLY
        cmdsize 48
[...]
more stuff for the dynamic link loader
[...]
Load command 5
     cmd LC_SYMTAB
 cmdsize 24
[...]
Load command 6
            cmd LC_DYSYMTAB
        cmdsize 80
[...]
Load command 7
          cmd LC_LOAD_DYLINKER
      cmdsize 32
         name /usr/lib/dyld (offset 12)
Load command 8
     cmd LC_UUID
 cmdsize 24
   uuid BA5465D2-66DA-8DD2-AB79-BE43982B15A3
Load command 9
        cmd LC_UNIXTHREAD
    cmdsize 184
[...]
Load command 10
          cmd LC_LOAD_DYLIB
      cmdsize 56
         name /usr/lib/libSystem.B.dylib (offset 24)
   time stamp 2 Thu Jan  1 01:00:02 1970
      current version 125.2.0
compatibility version 1.0.0
Load command 11
      cmd LC_CODE_SIGNATURE
  cmdsize 16
[...]

Find out if a Mach-O executable is prebound. If the executable is prebound, the word "PREBOUND" should be displayed in the flags section of the header.

nargothrond:~ --> otool -hv $(which otool)
/usr/bin/otool:
Mach header
      magic cputype cpusubtype  caps    filetype ncmds sizeofcmds      flags
MH_MAGIC_64  X86_64        ALL LIB64     EXECUTE    12       1872   NOUNDEFS DYLDLINK TWOLEVEL

lipo

From the man page: The lipo command creates or operates on "universal" (multi-architecture) files. It only ever produces one output file, and never alters the input file. The operations that lipo performs are:

• listing the architecture types in a universal file
• creating a single universal file from one or more input files
• thinning out a single universal file to one specified architecture type
• and extracting, replacing, and/or removing architectures types from the input file to create a single new universal output file

Listing

# lipo -info iphoneos/libboost_thread.a 
Architectures in the fat file: iphoneos/libboost_thread.a are: armv7 armv6 
# lipo -info iphonesimulator/libboost_thread.a 
input file iphonesimulator/libboost_thread.a is not a fat file
Non-fat file: iphonesimulator/libboost_thread.a is architecture: i386
# lipo -info /Applications/iTunes.app/Contents/MacOS/iTunes 
Architectures in the fat file: /Applications/iTunes.app/Contents/MacOS/iTunes are: ppc i386 

ar

Creates and maintains groups of files combined into an archive. Once an archive has been created, new files can be added and existing files can be extracted, deleted, or replaced.

Note: Static libraries are archives that were processed with ranlib.

Examples:

# Create an archive (or static library)
ar cru libFoobar.a foo.o bar.o
ranlib libFoobar.a

# Print a verbose list of the contents of an archive file
ar -tv /usr/local/src/fuego-0.4.1/go/libfuego_go.a 
rw-r--r--     502/20       123272 Jan 23 14:59 2011 __.SYMDEF
rw-r--r--     502/20       141720 Jan 23 14:58 2011 libfuego_go_a-GoBensonSolver.o
rw-r--r--     502/20       199856 Jan 23 14:58 2011 libfuego_go_a-GoBlock.o
rw-r--r--     502/20       269840 Jan 23 14:58 2011 libfuego_go_a-GoBoard.o
[...]

libtool / ranlib

libtool takes the specified input object files and creates a library for use with the link editor, ld.

ranlib (which is a symlink to libtool) adds or updates the table of contents of archive libraries produced with ar. To produce a static library:

ar cru libFoobar.a foo.o bar.o
ranlib libFoobar.a

arch

Prints the machine's architecture type, or runs the selected architecture of a universal binary. For instance:

nargothrond:~ --> arch
i386

A few other tools have an -arch argument that accepts an architecture. Valid architectures to choose from are (list taken from "man arch"):

• i386
• ppc
• ppc64
• x86_64

machine

Displays the machine type. For instance:

nargothrond:~ --> machine
i486

nm

Displays the symbol table of an object file (nm = name list).

% nm libfoobar.so.1
         U xxx
00010fe8 W yyy
00068718 T zzz
[...]

The meaning of some of the letter codes are:

• U = Undefined. Library uses the symbol, but does not define it
• W = Weak. Library defines the symbol, but allows it to be redefined somewhere else
• T = Text. Library defines the symbol in the Text (= Code) segment
• D = Data. Library defines the symbol in the Data segment
• I = Indirect symbol
• Refer to "man nm" for other symbol types

To display the symbols of a single object file within an archive

nm "libx.a(x.o)"

If an archive contains multiple architectures (i.e. a universal file, or "fat file"), then the following command can be used to display the symbols of only a specific architecture. By default nm displays only symbols from the host architecture, if the archive contains it; otherwise, it displays symbols for all architectures in the archive.

nm -arch arm64 boost.framework/boost

Display the symbols an object is importing (instead of the ones it defines itself), which library the symbols are imported from, and how the symbols are referenced.

• This does not tell us whether an imported symbol is actually present in the library that is the import source
• In fact, this doesn't even tell us whether the library that is the import source is actually present on the system
• These things can be found out only by actually executing a binary that includes the object file (maybe there are other ways, but at the moment I don't know them)
• What we can say from the list is whether a symbol is lazy bound (see section Lazy binding) or weakly linked (see section Weak linking)

nm -mg /Applications/0xED.app/Contents/MacOS/0xED
/Applications/0xED.app/Contents/MacOS/0xED (for architecture ppc):
00000000 (absolute) external .objc_category_name_NSBezierPath_STAddition
00000000 (absolute) external .objc_category_name_NSDictionary_STAddition
[...]
0004a3e0 (__OBJC,__class) external .objc_class_name_ASCIITablePanelController
[...]
00000000 (prebound undefined [lazy bound]) external .objc_class_name_NSAlert (from Cocoa)
[...]
00046000 (__DATA,__data) [referenced dynamically] external _NXArgc
[...]
0002cf44 (__TEXT,__text) external __Z11DVMemMemBckPKvmS0_m
[...]
0003192c (__TEXT,__text) weak external __ZN6Common19SpecialDiskFragment13SplitAtOffsetExPPNS_12FileFragmentES3_
[...]
00000000 (prebound undefined [lazy bound]) weak external __ZdaPv (from libstdc++)

strip

Besides other uses (not yet researched here), strip -c can be used to remove code from a dynamic library, leaving only the symbol table and thus creating a stub library output file. Stub libraries like this are useful for linking software against them, but not for executing programs. Mac OS X and iOS SDKs contain stub libraries.

Techniques

Lazy binding

Lazy binding, as explained in :

• Lazy binding occurs only for symbols that are functions ("non-variable symbols")
• Inspect symbols and their binding using nm -mg
• When a program is run, and it contains a lazy bound function, the library that contains the function is not loaded by dyld right away; rather, it is loaded when the function is actually executed
  • Of course, the library may be loaded when other functions in that library are used, in which case the lazy bound symbol still remains unbound
• The use of the function triggers a "fixup call" into dyld, where it then tries to resolve the symbol before execution jumps to the actual function in question
• Regular function calls usually result in lazy binding of that function
• If a function pointer is stored in a variable, however, the function is non-lazy bound, because dyld can't be assured of having a chance to resolve the symbol address before it is executed

foobar = InitCursor; // This forces InitCursor to be non-lazy
foobar();

• At some point in Technote 2064 , it is suggested that lazy bound symbols can be tested for existence by using Availability Macros. I strongly doubt that this is true since Availability Macros are later explained to depend on weak linking.

Weak linking

Most of the information that follows is taken from :

• Weak linking (together with Availability Macros) is available since Mac OS X 10.2. The environment variable MACOSX_DEPLOYMENT_TARGET (also a build setting in Xcode) tells the linker whether or not it can use weak linking
• The opposite of weak linking is strong linking
• Weak linking allows to link a symbol such that the symbol does not have to be present at runtime for the binary to continue running. Of course, the program can't actually try to use the symbol if it is non-existent, or it will crash.
• As opposed to lazy binding, a program can check to see if the address of a weak linked symbol is NULL. If it is, then the symbol is not available, but if the address is a real address, then the symbol exists and can be used
  • The exact way how weak linked things exist is explained with details in
• A symbol will be linked strongly unless its prototype (as seen at compile time) is explicitly marked as weak. Typically this is done in the header containing the prototype for the routine, and is done by adding the weak_import attribute to the prototype
• Apple frameworks and libraries use Availability Macros (see next chapter) for marking their prototypes as weak
• Usually weak linking occurs for individual symbols, however it is also possible to weak link an entire framework or library. If all symbols that an application imports from a given framework or library are weakly linked, then the linker will automatically mark the framework or library weak as a whole in the application binary
• Use "nm -mg" to examine a binary for weak linking of individual symbols
• Use "otool -l" to look for entire frameworks or libraries being weak linked
• Because weak linking was introduced in Mac OS X 10.2, weak linking cannot be used to make an application run on Mac OS X versions prior to 10.2. Weakly linked symbols will be seen as strongly linked symbols on 10.1.x, and weakly linked frameworks or shared libraries will cause the application to crash at launch on 10.1.x.
• Weak linking has been designed to solve problems with strongly typed languages such as C or C++. Dynamically typed Objective-C is not affected as long as one checks for the existence of newly introduced Objective-C methods and classes at runtime and avoids their use as appropriate

Availability Macros

From and :

• /usr/include/Availability.h and /usr/include/AvailabilityMacros.h help to determine which OS versions introduced the APIs you are using, and tells the compiler which routines should be weakly linked
  • Availability.h is for targeting iOS and Mac OS X 10.6 and later
  • AvailabilityMacros.h is for targeting Mac OS X 10.2 - 10.5
• Apple frameworks and libraries automatically use Availability Macros for marking their prototypes as weak. These macros do not need to be included in your project, they are automatically included by the header file which contains a prototype marked as weak
• To prepare your own frameworks and libraries for weak linking, you should also use Availability Macros. I haven't researched how exactly to do this yet.
• To influence the way how the macros work in your project, set the precompiler macros MAC_OS_X_VERSION_MIN_REQUIRED and MAC_OS_X_VERSION_MAX_ALLOWED to define the range of OS versions that your application will run with. This must be done before Availability.h and/or AvailabilityMacros.h are included (i.e. define the macros as a project-wide setting).
• If the variables are undefined, the following rules apply
  • MAC_OS_X_VERSION_MIN_REQUIRED is set to 10.0, unless the environment variable MACOSX_DEPLOYMENT_TARGET is set, in which case MAC_OS_X_VERSION_MIN_REQUIRED is set to the environment variable's value
  • MAC_OS_X_VERSION_MAX_ALLOWED is set to the highest major OS version that AvailabilityMacros.h is aware of. When you you use an SDK, this will be the latest version of the OS for which the SDK has been released
• Set MAC_OS_X_VERSION_MAX_ALLOWED to the same value as MAC_OS_X_VERSION_MIN_REQUIRED to check which APIs are used by the application that are not available on what one thinks is the minimum system requirement
• In practice, one will usually not care about all this because the defaults are set up in a sensible way!

Prebinding

The source of the information in this chapter is .

• Prebinding is the process of computing the addresses for symbols imported by a shared library or application prior to their use
• Resolving these addresses before their use reduces the amount of work performed by the dynamic loader (dyld) at runtime and results in faster launch times for applications
• In Mac OS X 10.4, dyld was improved in a way that eliminated the need for prebinding information in most situations. The system libraries are now the only executables that are still prebound, and they are prebound in a way that optimizes performance on the target system. Because of the improved prebinding of the system libraries, applications and third-party libraries no longer need to be prebound
• When developing applications for versions of Mac OS X prior to 10.4, prebinding is considered optional
• Changes in Mac OS X 10.3.4 made application prebinding unnecessary but applications running on earlier versions of the operating system still received some benefits from prebinding. If you feel your application launches slowly on pre-10.3.4 systems, build your application prebound and see if launch time improves.
• To determine if a Mach-O executable is prebound, use otool -hv" to view the executable's Mach header information. If the executable is prebound, the word "PREBOUND" should be displayed in the flags section of the header

Symbol visibility

This section is about how to control visibility of symbols in a shared library, as seen from the outside.

• It is useful to define some symbols to be visible within the entire library, but not outside of it, in order to keep the library interface small
  • And also because libraries with smaller interfaces have faster load times
• Prior to gcc 4.0 there were two techniques, of which I am not sure how exactly they work
  • Declare something using the keyword __private_extern__
  • Making an "export list", which is a file containing the names of symbols that should explicitly be hidden or shown
• From gcc 4.0 onwards, there are the following new methods for controlling symbol visibilty:
  • Using the compiler option -fvisibility=foo for an entire compilation unit
  • Using the compiler option -fvisibility-inlines-hidden for an entire compilation unit
  • Defining the visibility attribute in code for a single symbol: __attribute__((visibility("foo")))
  • Using a pragma to define the visibility attribute for all symbols within a block of code: #pragma GCC visibility push(foo) followed by #pragma GCC visibility pop
• Possible visibility settings (both for the compiler option and the visibility attribute) are
  • default
  • hidden
  • If visibility is not specified, the compiler assumes default visibility
• The visibility attribute definition overrides usage of the compiler option
• Visibility may be applied to functions, variables, templates, and C++ classes
• If a class is marked as hidden, all of its member functions, static member variables, and compiler-generated metadata, such as virtual function tables and RTTI information, are also hidden
• Apparently, visibility of inline functions is quite an issue, hence the possibility to make them all hidden in one fell swoop using the -fvisibility-inlines-hidden compiler option (which it is recommended to use)

An example:

int a(int n) {return n;}
__attribute__((visibility("hidden"))) int b(int n) {return n;}
__attribute__((visibility("default"))) int c(int n) {return n;}

class X
{
public:
  virtual ~X();
};
class __attribute__((visibility("hidden"))) Y
{
public:
  virtual ~Y();
};
class __attribute__((visibility("default"))) Z
{
public:
  virtual ~Z();
};

void f() { }
#pragma GCC visibility push(default)
void g() { }
void h() { }
#pragma GCC visibility pop

#define EXPORT __attribute__((visibility("default")))
EXPORT int foo();

The following table shows how using the -fvisibility=foo compiler option affects visibility of symbols in the above code:

Symbol name	Visibility attribute in code	No compiler option	-fvisibility=default	-fvisibility=hidden	Remarks
Function a()	none (implicitly default)	visible	visible	hidden
Function b()	hidden	hidden	hidden	hidden
Function c()	default	visible	visible	visible
Class X	none (implicitly default)	visible	visible	hidden
Class Y	hidden	hidden	hidden	hidden
Class Z	default	visible	visible	visible
Function f()	none (implicitly default)	visible	visible	hidden
Function g()	default	visible	visible	visible
Function h()	default	visible	visible	visible
Function foo()	default	visible	visible	visible	Note the usage of the preprocessor macro. This may be put to use similarly to the "__declspec(dllexport)" idiom used on Windows

Final notes:

• If not explicitly set differently, an Xcode project sets the compiler options -fvisibility=hidden and -fvisibility-inlines-hidden. The consequences are:
  • If the project is a shared library, it will export nothing unless a symbol is explicitly marked visible using the visibility attribute in the code
  • Whenever the project includes/imports headers from an external library, the symbols used from that library must be marked visible, again using the visibility attribute in the code. It can be assumed that Apple's frameworks and system libraries appropriately employ the visibility attribute
  • Consequently, if an external library is used that does not have visibility support built in (i.e. it does not use the visibility attribute in its code), it must be assumed that the library has been built without any of the visibility compiler options, which results in all symbols being exported. The project's compiler options should therefore be set to -fvisibility=default and not use -fvisibility-inlines-hidden, otherwise the project will use the symbols but declare them hidden - while the external library will declare all of its symbols visible. The probable result will be at least a compiler warning ("foo has different visibility (default) in /path/to/shared-lib and (hidden) in /path/to/project-object-file").
  • Since the external library does not use the visibility attribute in its code, it must be assumed that it has also been built without any thought towards using the visibility compiler options
• The pragma might be handy to hide the private members of a class
• Exceptions and classes that are used with dynamic_cast must always be exported
• The article on gnu.org (referenced below) has additional information about some complexities
• Further research is needed to find out how all this interacts with static and extern declarations (e.g. traditionally in C, to make a symbol invisibly outside its compilation unit (the .c file), it is declared static)

References:

• C++ Runtime Environment: Controlling Symbol Visibility (Mac OS X Reference Library)
• Symbol Visibility (GCC Wiki)

Dynamic library loading

Dynamic library loading is done by dyld at runtime when a program needs to load a shared library or a framework. Typically this occurs when the program is started, but it may also occur well after that time (e.g. when lazy bound symbols are resolved.

A mach-o binary (e.g. an executable program, or a shared library) stores a list of shared libraries and frameworks that it depends on. Use otool to obtain this list:

otool -L /path/to/binary-file

The list printed by otool contains paths, however these may not be the real paths used by dyld to load the shared libraries or frameworks. Here's an example that demonstrates this:

caradhras:~ --> otool -L /Applications/Gimp.app/Contents/MacOS/gimp-2.8 
/Applications/Gimp.app/Contents/MacOS/gimp-2.8:
	/tmp/skl/Gimp.app/Contents/Resources/lib/libgimpwidgets-2.0.0.dylib (compatibility version 801.0.0, current version 801.10.0)
	/tmp/skl/Gimp.app/Contents/Resources/lib/libgimpconfig-2.0.0.dylib (compatibility version 801.0.0, current version 801.10.0)
	/tmp/skl/Gimp.app/Contents/Resources/lib/libgtkmacintegration.2.dylib (compatibility version 3.0.0, current version 3.0.0)
[...]

The path /tmp/skl does not exist on my system, so clearly dyld must be capable of finding the required libraries/frameworks in some other way. The complete documentation how this works is available via

man dyld

Here is an overview of the most important parts of the mechanism:

• Frameworks only: Search in the paths specified by the DYLD_FRAMEWORK_PATH environment variable
• Both frameworks/libraries: Search in the paths specified by the DYLD_LIBRARY_PATH environment variable
• Both frameworks/libraries: Try to use the path as it is stored in the mach-o binary
  • This path consists of what is called the "install path" and the "install name"
  • For instance, /usr/lib/libSystem.B.dylib has an install path = /usr/lib and an install name = libSystem.B.dylib.
• Frameworks only: Search in the paths specified by the DYLD_FALLBACK_FRAMEWORK_PATH environment variable. If not set, this has the default /Library/Frameworks:/Network/Library/Frameworks:/System/Library/Frameworks.
• Both frameworks/libraries: Search in the paths specified by the DYLD_FALLBACK_LIBRARY_PATH environment variable. If not set, this has the default $(HOME)/lib:/usr/local/lib:/lib:/usr/lib.

Besides the use of these environment variables, there are options how the "install path" can be made variable:

• Instead of encoding an absolute path, the variable prefix @executable_path can be used. This causes dyld to look for the shared library or framework relative to the path of the main executable of the process. For instance, if the executable is /Applications/Foo.app/Contents/MacOS/foo, and the install path is @executable_path/../Frameworks/Bar.framework/Versions/A/bar, then the referenced framework "Bar" will be found inside the application bundle regardless of where in the filesystem the bundle is located!.
• A similar variable prefix is @loader_path. This causes dyld to look for the shared library or framework relative to the path of the mach-o binary that contains the install path. This is important for things like plugins which are not an executable program, but are instead loaded by an executable program into an already existing process. A plugin bundle that uses @loader_path can be stored anywhere in the filesystem, and the referenced framework or library will be found regardless of which executable loads the plugin.
• Finally, the variable prefix @rpath. This is a bit more complicated, so it gets it own paragraph...

When ld creates a mach-o binary, it is possible to tell it to embed so-called "run paths" into the binary. This is done by using the option -rpath, for instance:

ld -rpath /some/path
ld -rpath @executable_path/Frameworks
ld -rpath @loader_path/Frameworks
# If clang is used as the front-end for the linker
clang -Xlinker -rpath /some/path

You can see the run paths embedded in a binary with otool -l, they will show up as load commands of type LC_RPATH. Now here is how dyld deals with these run paths:

• When dyld loads a shared library or framework it scans that library's/framework's run paths
• Moreover, dyld maintains an entire stack of run paths it has encountered in the entire dependency chain. For instance, executable A is run, which depends on shared library B, which in turn depends on framework C. At the time when dyld attempts to load framework C it has remembered all the run paths embedded in executable A and shared library B.
• When dyld needs to load a shared library or framework, and that library or framework is marked with the @rpath variable prefix, then dyld substitutes the prefix with all the run paths that it has encountered so far, until it can find the library or framework
• When a run path contains @loader_path, that variable is replaced by the path of the mach-o binary that contains the run path

SDK-based Development, or Cross-Development

References

The two main sources for the information in this chapter are

• SDK Compatibility Guide
• Technote 2064

Conclusions

The conclusions I draw from researching the matter are these:

• The Deployment Target should always be the same as the base SDK, unless my projects check for undefined function calls due to weak linking
• When building in Xcode, this already is the default behaviour
• When building a project without Xcode (e.g. using the GNU Autotools), the Deployment Target must be set explicitly
• To compile code for Intel Macs, at least the 10.4u SDK and gcc 4.0 is required

Concepts

• Cross-development depends on
  • Weak linking (introduced in Mac OS X 10.2)
  • The presence of SDKs (Mac OS X SDKs were introduced in Mac OS X 10.3)
  • Cross-Development support (first available in the Xcode Tools distributed with Mac OS X 10.3)
• An SDK (software development kit) represents a specific version of an operating system
  • For Mac OS X, SDKs refer to major OS numbers (e.g. MacOSX10.6.sdk) but actually represent the latest minor release of that OS version
  • For iOS, SDKs refer to minor OS numbers (e.g. iPhoneOS4.2.sdk, iPhoneSimulator4.2.sdk)
• At the time of writing, SDKs locations are these
  • Mac OS X SDKs: /Developer/SDKs
  • iOS: /Developer/Platforms/iPhoneOS.platform/Developer/SDKs/
  • iPhone Simulator: /Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/
• It does not matter what version of Mac OS X is running on the computer that is used to develop software - what is important is the SDK that the software is built against, and how the build is set up. There are two ways of developing and building software
  • Build software for one version of an OS and forward-compatible with later versions. The software does not use newer features.
  • Build software for a range of OS versions, so that it can run on older versions but can take advantage of features in newer ones.
• Besides targetting versions of Mac OS X different from the development system, this concept also allows to cross-develop software for entirely different system types (iOS at the time of writing)

Anatomy of an SDK

• An SDK basically contains the complete set of header files and stub libraries that shipped for the OS release it represents
  • The stub libraries are for linking only; they do not contain executable code but just the exported symbols
  • This allows SDKs to take up far less disk space and reduces the possibility that an SDK library will be loaded at runtime
  • Only system dynamic libraries and frameworks are provided in stub library form. Static libraries (such as those required by GCC) are provided in the SDK and included in your code as if they have been drawn from the /usr/lib directory of the operating system being targeted
• An SDK also contains the compiler toolchain required to build software against the SDK
• An .sdk directory resembles the directory hierarchy of the OS release it represents
• For instance, it has usr, System, and Developer directories at its top level
• Mac OS X .sdk directories also contain a Library directory
• Each of these directories in turn contains subdirectories with the headers and libraries that are present in the corresponding version of the operating system with Xcode installed.

Base SDK and Deployment Target

The build settings to use cross-development are:

• Base SDK
  • The software is built as though it were built in that version of the operating system
  • The software will be optimized for that version of the OS, that is, it will be "prebound" to the standard locations of the frameworks in that version. See "Prebinding" below.
  • By default, Xcode sets this to the newest OS supported by Xcode
  • The environment variable for this is SDKROOT
• Deployment Target
  • This specifies the earliest OS version on which the software will run
  • By default, Xcode sets this to the version of the OS corresponding to the base SDK version and later
  • The environment variables for this currently are MACOSX_DEPLOYMENT_TARGET and IPHONEOS_DEPLOYMENT_TARGET
• Together, these settings define the range of OS versions from which you can use features
  • The software can unconditionally use features from OS versions up to and including the system version that was specified as the Deployment Target
  • The software can use features from OS versions later than the Deployment Target - up to and including the OS version that was selected as the base SDK - but the software must check for the availability of each feature (e.g. by checking each function pointer for a NULL value before calling it).
• The exact way how to check if weak linked features exist, is explained with details in

                                                    Deployment                Base
                                                    Target                    SDK

10.0  -----  10.1  -----  10.2  -----  10.3  -----  10.4  -----  10.5  -----  10.6  -----  ???

-------------------------------------------------------------->  ----------------------->  ------------>
 unconditionally use APIs from these OS versions                 conditionally use         do not use
                                                                 these APIs                these APIs

 strong linking                                                  weak linking              no linking

Additional notes on MACOSX_DEPLOYMENT_TARGET:

• MACOSX_DEPLOYMENT_TARGET is an environment variable, which can also be set as a build setting within an Xcode project
• The variable was introduced in 10.2 to make the "weak linking" feature possible
• If the variable is not set, compiler/linker assume a value of 10.1. In reality this applies only to builds that are made without Xcode, because Xcode automatically assigns a value matching the base SDK

Additional notes on base SDK:

• If the same code base is used to target different base SDKs, the code must probably use conditional compiling based on Availability Macros. Chapter "Conditionally Compiling for Different SDKs" in the SDK Compatibility Guide explains how this is done.

Build settings in Xcode

• The SDK settings in the project inspector control the value of a build setting called the Base SDK, or SDKROOT
• This build setting points to the root directory in which to find headers and libraries to build against
• If SDKROOT is not set, a project is compiled against the headers and linked against the runtime libraries in the current system (/usr/include, /usr/lib, and /System/Library/Frameworks)
• The AvailabilityMacros.h header found in SDKROOT sets the preprocessor macro MAC_OS_X_VERSION_MAX_ALLOWED to a constant that is equivalent to the version represented by the SDK
• The MACOSX_DEPLOYMENT_TARGET or IPHONEOS_DEPLOYMENT_TARGET build setting is used by the compiler to enable weak linking. The compiler also uses this value to set the preprocessor symbol MAC_OS_X_VERSION_MIN_REQUIRED
• When you select an SDK for a project, the SDK usually applies to all targets of the project. It is possible to specify an SDK on a per-target basis, but this may interfere with some Xcode features that expect a project-wide SDK setting (e.g. code completion)
• The Deployment Target can be set either for an individual target, or for all targets of the project
• Set the SDK in the "project inspector" window, which can be displayed by selecting the project in the "Groups & Files" pane on the left of the Xcode window, then typing Cmd+I
• Set the Deployment Target for all targets: Project Inspector window > Build Tab. By default the deployment target is set to the version that matches the SDK.

Build settings in Makefiles

In a makefile-based project (often based on the Autotools), SDK build settings occur via environment variables :

• Using SDKs requires GCC 4.0 or later
• To choose an SDK, specify the full path to the desired SDK directory for the following options:
  • Use the -isysroot option with the compiler. This is typically specified using the environment variable CPPFLAGS.
  • Use the -syslibroot option with the linker. This is typically specified using the environment variable LDFLAGS.
• Set the deployment target as follows:
  • Use the environment variable MACOSX_DEPLOYMENT_TARGET for Mac OS X builds
  • Use the environment variable IPHONEOS_DEPLOYMENT_TARGET for iOS builds

Intel vs. PowerPC

• The minimum requirement for producing Intel code is the 10.4u SDK and the gcc 4.0 compiler
  • As a side note: The "u" suffix in "10.4u" indicates the capability of the SDK to produce universal binaries. The suffix distinguishes the SDK from the original 10.4 SDK which was PowerPC only. The original SDK no longer ships with Xcode, but can be downloaded from the ADC member site (unclear if this is still the case)
• From an API-based point of view, an application might use only "old" features that already existed before Mac OS X 10.4. In such a situation, it might be desirable to produce a PowerPC binary with an SDK of 10.3 or earlier. This is made possible in Xcode by build settings that can be set to different values depending on the architecture (e.g. SDKROOT)
• See , chapter "Cross-Development and Universal Binaries" for details about configuring architecture specifics and still producing a universal binary; the same chapter also gives hints about producing a universal binary from the command line
• The guide for creating universal binaries is ; this document also contains an appendix that explains the Rosetta technology

Xcode 4.5 environment

Folders

• Xcode 4.5 is installed in /Applications/Xcode.app. Xcode 4.2 and older versions were in /Developer/Applications/Xcode.app.
• Xcode 4.2 and older versions had their platforms in /Developer/Platforms. With Xcode 4.5, all the compiler binaries, libraries, SDKs etc. are now located inside the app bundle, below platform-specific folders:

# l /Applications/Xcode.app/Contents/Developer/Platforms/
total 0
drwxr-xr-x@ 5 admin  admin  170  8 Sep 02:40 .
drwxr-xr-x@ 9 admin  admin  306  8 Sep 02:48 ..
drwxr-xr-x@ 8 admin  admin  272  8 Sep 02:44 MacOSX.platform
drwxr-xr-x@ 9 admin  admin  306  8 Sep 02:52 iPhoneOS.platform
drwxr-xr-x@ 6 admin  admin  204  8 Sep 02:58 iPhoneSimulator.platform

• The only other difference regarding folder structure between Xcode 4.5 and 4.2 is that the MacOSX platform folder now also contains an SDK subfolder. Strangely, in Xcode 4.2 the SDKs for the MacOSX platform were located in /Developer/SDKs/.
• Finally, by default Xcode 4.5 does not install any of the command line developer tools (e.g. gcc) into the global folder /usr/bin.

Multiple Xcode installations

Since Xcode is now self-contained and the developer tools are no longer globally installed in /usr/bin (more on that later), it is really easy to have multiple versions of Xcode installed in parallel. Switching between them is done via the xcode-select command line tool.

• Print the currently active Xcode version:

# xcode-select -print-path
/Applications/Xcode.app/Contents/Developer

• Switching to another Xcode version

xcode-select -switch /Applications/Xcode4.4.app

• Use the environment variable DEVELOPER_DIR to switch only temporarily for the current shell session:

export DEVELOPER_DIR="/Applications/Xcode4.4.app"

Running command line tools

The currently active Xcode version (see previous section on xcode-select) affects from where the command line tools are run.

• Build an Xcode project

xcodebuild /path/to/Foo.xcodeproj

• Run a specific build tool, in this example gcc. The SDK in the SDKROOT environment variable is used (can be overridden with the -sdk argument).

xcrun gcc [options]

• Don't run the tool, just display the location where it is run from

xcrun -find gcc

Installing global command line developer tools

By default, Xcode 4.5 does not install any developer tools to the global location /usr/bin. However, projects such as Fink or MacPorts require that dev tools are available. The simplest solution is to install the tools from witihin Xcode, under "Preferences > Download > Components".

Xcode 4.3 and 4.4 environment

I skipped these versions of Xcode, so I don't really know much about them. Presumably Xcode 4.3 started the practice to place everything inside the Xcode.app bundle, i.e.

/Applications/Xcode.app/Contents/Developer

Xcode 4.2 environment

When Xcode 4.2 (and possibly older versions of Xcode 4) is installed together with an SDK of some sort, the following environment is created:

• Everything is installed under /Developer
• A number of applications (including Xcode) and goodies exists under /Developer/Applications and /Developer/Extras
• Platforms are located under /Developer/Platforms:

# l /Developer/Platforms/
total 0
drwxrwxr-x   5 root  admin  170 26 Jan 17:00 .
drwxrwxr-x@ 16 root  admin  544 26 Jan 17:04 ..
drwxrwxr-x   8 root  admin  272 26 Jan 16:58 MacOSX.platform
drwxrwxr-x   7 root  admin  238 26 Jan 17:01 iPhoneOS.platform
drwxrwxr-x   5 root  admin  170 26 Jan 17:01 iPhoneSimulator.platform

• Depending on how much specifics each platform has, more or less of the basic directory structure under /Developer is repeated below each platform's base directory:

# l /Developer/Platforms/iPhoneOS.platform/Developer/
total 0
drwxrwxr-x  8 root  admin  272 26 Jan 17:01 .
drwxrwxr-x  7 root  admin  238 26 Jan 17:01 ..
drwxrwxr-x  3 root  admin  102 12 Okt 06:11 Applications
drwxrwxr-x  3 root  admin  102  5 Aug  2010 Documentation
drwxrwxr-x  5 root  admin  170 26 Jan 17:00 Library
drwxr-xr-x  3 root  wheel  102 27 Okt 05:03 SDKs
drwxrwxr-x  3 root  admin  102 12 Okt 06:57 Tools
drwxrwxr-x  8 root  admin  272 26 Jan 17:01 usr
# l /Developer/Platforms/iPhoneSimulator.platform/Developer/
total 8
drwxrwxr-x  7 root  admin  238 26 Jan 17:01 .
drwxrwxr-x  5 root  admin  170 26 Jan 17:01 ..
drwxrwxr-x  3 root  admin  102 22 Okt 08:02 Applications
drwxrwxr-x  6 root  admin  204 22 Okt 07:35 Library
drwxr-xr-x  6 root  wheel  204 26 Jan 17:02 SDKs
drwxrwxr-x  3 root  admin  102 12 Okt 06:57 Tools
lrwxr-xr-x  1 root  admin   12 26 Jan 17:01 usr -> ../../../usr
# l /Developer/Platforms/MacOSX.platform/Developer/
total 0
drwxrwxr-x  3 root  admin  102 18 Mai  2009 .
drwxrwxr-x  8 root  admin  272 26 Jan 16:58 ..
drwxrwxr-x  5 root  admin  170 26 Jan 16:58 Library

• Each platform may have its own SDKs. Note that the Mac OS X platform SDKs are located directly under /Developer:

# l /Developer/Platforms/iPhoneOS.platform/Developer/SDKs/
total 0
drwxr-xr-x  3 root  wheel  102 27 Okt 05:03 .
drwxrwxr-x  8 root  admin  272 26 Jan 17:01 ..
drwxr-xr-x  8 root  wheel  272 27 Okt 10:02 iPhoneOS4.2.sdk
# l /Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/
total 0
drwxr-xr-x  6 root  wheel  204 26 Jan 17:02 .
drwxrwxr-x  7 root  admin  238 26 Jan 17:01 ..
drwxr-xr-x  8 root  wheel  272 26 Jan 17:03 iPhoneSimulator3.2.sdk
drwxr-xr-x  9 root  wheel  306 23 Sep 06:14 iPhoneSimulator4.0.sdk
drwxr-xr-x  8 root  wheel  272 18 Aug  2010 iPhoneSimulator4.1.sdk
drwxr-xr-x  8 root  wheel  272 22 Okt 08:00 iPhoneSimulator4.2.sdk
# l /Developer/SDKs/
total 0
drwxr-xr-x   5 root  wheel  170 26 Jan 16:59 .
drwxrwxr-x@ 16 root  admin  544 26 Jan 17:04 ..
drwxr-xr-x   7 root  wheel  238 18 Mai  2009 MacOSX10.4u.sdk
drwxr-xr-x   7 root  wheel  238 24 Jun  2010 MacOSX10.5.sdk
drwxr-xr-x   7 root  wheel  238  5 Okt 02:40 MacOSX10.6.sdk

• Each platform has its own compiler/linker toolchain. Note that the Mac OS X platform's toolchain is also present under /usr/bin (appears to be a copy of what's in /Developer/usr/bin).

# l /Developer/Platforms/iPhoneOS.platform/Developer/usr/bin/gcc*
lrwxr-xr-x  1 root  admin      7 26 Jan 17:01 /Developer/Platforms/iPhoneOS.platform/Developer/usr/bin/gcc -> gcc-4.2
-rwxrwxr-x  1 root  admin  42160  5 Aug  2010 /Developer/Platforms/iPhoneOS.platform/Developer/usr/bin/gcc-4.0
-rwxrwxr-x  1 root  admin  51440  5 Aug  2010 /Developer/Platforms/iPhoneOS.platform/Developer/usr/bin/gcc-4.2
# l /Developer/Platforms/iPhoneSimulator.platform/Developer/usr/bin/gcc*
lrwxr-xr-x  1 root  wheel       7 26 Jan 17:00 /Developer/Platforms/iPhoneSimulator.platform/Developer/usr/bin/gcc -> gcc-4.2
-rwxr-xr-x  1 root  wheel   97392 21 Jul  2010 /Developer/Platforms/iPhoneSimulator.platform/Developer/usr/bin/gcc-4.0
-rwxr-xr-x  1 root  wheel  166128 24 Jun  2010 /Developer/Platforms/iPhoneSimulator.platform/Developer/usr/bin/gcc-4.2
# l /Developer/usr/bin/gcc*
lrwxr-xr-x  1 root  wheel       7 26 Jan 17:00 /Developer/usr/bin/gcc -> gcc-4.2
-rwxr-xr-x  1 root  wheel   97392 21 Jul  2010 /Developer/usr/bin/gcc-4.0
-rwxr-xr-x  1 root  wheel  166128 24 Jun  2010 /Developer/usr/bin/gcc-4.2
# l /usr/bin/gcc*
lrwxr-xr-x  1 root  wheel       7 26 Jan 17:06 /usr/bin/gcc -> gcc-4.2
-rwxr-xr-x  1 root  wheel   97392 21 Jul  2010 /usr/bin/gcc-4.0
-rwxr-xr-x  1 root  wheel  166128 24 Jun  2010 /usr/bin/gcc-4.2

• As with platforms, different SDKs may have their own specifics which will be reflected by a repeat of the /Developer directory structure below each SDK's base directory:

# l /Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS4.2.sdk/Developer/
total 0
drwxr-xr-x  4 root  wheel  136 27 Okt 10:02 .
drwxr-xr-x  8 root  wheel  272 27 Okt 10:02 ..
drwxr-xr-x  4 root  wheel  136 27 Okt 10:03 Library
drwxr-xr-x  4 root  wheel  136 26 Jan 17:00 usr
# l /Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator4.2.sdk/Developer/
total 0
drwxr-xr-x  4 root  wheel  136 26 Jan 17:01 .
drwxr-xr-x  8 root  wheel  272 22 Okt 08:00 ..
drwxr-xr-x  4 root  wheel  136 26 Jan 17:01 Library
drwxr-xr-x  3 root  wheel  102 22 Okt 07:58 usr
# l /Developer/SDKs/MacOSX10.6.sdk/Developer/
total 0
drwxr-xr-x  4 root  wheel  136  5 Okt 02:39 .
drwxr-xr-x  7 root  wheel  238  5 Okt 02:40 ..
drwxr-xr-x  3 root  wheel  102 18 Mai  2009 Headers
drwxr-xr-x  3 root  wheel  102  5 Okt 02:40 usr

• SDKs also have other specific binaries and libraries outside of their respective Developer directory. It is not entirely clear which directory structures may be repeated at which level
• An interesting location is the SDK-specific compiler stuff. It appears that the name of each sub-directory can be used for the --host parameter to configure when cross-compiling in a GNU Autotools project

# l /Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS4.2.sdk/usr/lib/gcc
total 0
drwxr-xr-x   3 root  wheel   102  5 Aug  2010 .
drwxr-xr-x  94 root  wheel  3196 26 Jan 17:00 ..
drwxr-xr-x   4 root  wheel   136 27 Okt 10:03 arm-apple-darwin10
# l /Developer/Platforms/iPhoneSimulator.platform/Developer/SDKs/iPhoneSimulator4.2.sdk/usr/lib/gcc
total 8
drwxr-xr-x    4 root  wheel   136 26 Jan 17:01 .
drwxr-xr-x  109 root  wheel  3706 26 Jan 17:01 ..
drwxr-xr-x    4 root  wheel   136 26 Jan 17:00 i686-apple-darwin10
lrwxr-xr-x    1 root  wheel    19 26 Jan 17:00 i686-apple-darwin9 -> i686-apple-darwin10
# l /Developer/SDKs/MacOSX10.6.sdk/usr/lib/gcc
total 0
drwxr-xr-x    5 root  wheel    170 26 Jan 16:59 .
drwxr-xr-x  354 root  wheel  12036 26 Jan 16:59 ..
drwxr-xr-x    2 root  wheel     68 24 Jun  2010 arm-apple-darwin10
drwxr-xr-x    4 root  wheel    136  5 Okt 02:40 i686-apple-darwin10
drwxr-xr-x    4 root  wheel    136  5 Okt 02:40 powerpc-apple-darwin10
# l /Developer/SDKs/MacOSX10.5.sdk/usr/lib/gcc
total 0
drwxr-xr-x    5 root  wheel    170 26 Jan 16:59 .
drwxr-xr-x  349 root  wheel  11866 26 Jan 16:59 ..
drwxr-xr-x    2 root  wheel     68 24 Jun  2010 arm-apple-darwin10
drwxr-xr-x    4 root  wheel    136 26 Jan 16:59 i686-apple-darwin10
drwxr-xr-x    4 root  wheel    136 26 Jan 16:59 powerpc-apple-darwin10
# l /Developer/SDKs/MacOSX10.4u.sdk/usr/lib/gcc
total 0
drwxr-xr-x    6 root  wheel   204 26 Jan 16:59 .
drwxr-xr-x  226 root  wheel  7684 26 Jan 16:59 ..
drwxr-xr-x    2 root  wheel    68 24 Jun  2010 arm-apple-darwin10
drwxr-xr-x    3 root  wheel   102 15 Mai  2009 darwin
drwxr-xr-x    3 root  wheel   102 24 Jun  2010 i686-apple-darwin10
drwxr-xr-x    3 root  wheel   102 24 Jun  2010 powerpc-apple-darwin10

Mac OS X / iOS platform vs. GNU Autotools

Refer to section about Makefile-based projects further up under SDK-based development.

• If CXX is left undefined, configure will abort at some stage due to a linker error (libgcc is not found). The reason for this is that configure finds, and uses, a compiler in /usr/bin, but this compiler does not work with the compiler flag -isysroot.
• CXX must refer to a compiler whose name explicitly identifies it as a C++ compiler (e.g. g++, clang++); if it refers to a compiler whose name identifies it as a C compiler (e.g. gcc), configure will abort at some stage due to an "undefined symbol" linker error. The reason for this is that CXX is used to compile C++ files; when a C++ file is compiled, a symbol is added to the object file which requires the C++ Standard Library at link time (libstdc++ by default, but may also be something else if the compiler/linker option -stdlib is specified). A compiler whose name identifies it as a C++ compiler adds the C++ Standard Library automatically to the linker step, while a C compiler does not.
• To use the new C++ Standard Library from the LLVM project, both the compiler AND the linker options must contain -stdlib=libc++. Since the purpose of libc++ is to support the C++11 standard, it is probably advisable to also use the compiler-only option -std=c++11.
• If several projects are built, all of them must use the same compiler name, otherwise there may be errors when the final link step occurs. An example is when clang and clang++ are mixed: This will produce a number of linker warnings in the final link step (the warnings are misleading, they warn about different visibility settings).
• Deployment target
  • This is handled via environment variables
  • Because the variables are not used in the construction of a command line, they must be exported!
  • They must also be present both for the "configure" and the "make" step
  • IPHONEOS_DEPLOYMENT_TARGET is for iPhoneOS builds
  • MACOSX_DEPLOYMENT_TARGET is for MacOSX builds
  • There is no variable for the simulator, it uses the one for iPhoneOS
• To create fat binaries (i.e. multiple architectures), simply specify multiple -arch flags to the compiler and the linker. Of course, the SDK must support the desired architecture.
• Base SDK
  • Use a compiler from the platform; e.g. /Developer/Platforms/iPhoneOS.platform/Developer/usr/bin/g++-4.2
  • Specify the base SDK using -isysroot; e.g. -isysroot /Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS4.2.sdk
  • Use -isysroot both for the compiler and the linker; -syslibroot for the linker does not work at present
    • Found on http://code.google.com/p/iphone-dev/wiki/Building: "-Wl,-syslibroot,/Your/SDK/Path does not work, but llvm-gcc will transform -isysroot to -syslibroot and override the one that's hardcoded into the binary."
• Configure flags --disable-shared and --enable-static for iPhoneOS builds apparently are not universally recognized
• --host configure flag
  • flag is for cross compiling, which causes configure to skip a few tests (notably it does not try to execute a program that was cross-compiled)
  • flag must match one of /Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS4.2.sdk/usr/lib/gcc
  • setting --build instead of --host does not work -> what's the difference?
• Compiler/linker errors such as the following may indicate a problem with the specified architecture (e.g. no architecture, or one that is not supported on the platform)

ld: library not found for -lcrt1.10.6.o
ld: library not found for -lgcc_s.10.5

Porting UNIX/Linux applications to Mac OS X

At the moment, this section exists only to provide space for a reference to the document "Porting UNIX/Linux applications to Mac OS X" .

Symbolicating crash logs

Basics

• Symbolicating is the process of analyzing a raw crash log and translating the numbers inside the log into meaningful references to lines of source code. Using these references it is possible to analyze the source code and track down the source of application crashes.
• In order for symbolicating to work, the following preconditions must be met:
  1. The application binary that produced the crash log must have been built with debug information in the format "DWARF with dSYM File". In Xcode, the setting that determines the debug information format is found under "Build Options".
  2. The debug information must be present as a .dSYM bundle on the machine where you attempt to symbolicate a crash log.
• If you attempt symbolicating on the same machine where you also prepared the build of your application that you distributed to your users, then you probably already have the appropriate .dSYM bundle. Here is why:
  • When you prepare a build for distribution, you usually select "Product > Archive" in Xcode
  • Doing this creates an archive bundle (extension .xcarchive) that is usually placed somewhere safe outside of your project's build folder. The archive bundle is, well, archived :-)
  • The debug information generated during the build is retained as a .dSYM sub-bundle inside the archive bundle
  • So in effect, when you later try to to symbolicate on the same machine where you created the archive for distribution, you are practically guaranteed to have the .dSYM bundle ready for the symbolicating process
• If you attempt symbolicating on a different machine, you need to copy the appropriate .dSYM bundles to this new machine
  • Knowing from above that .dSYM bundles are retained as sub-bundles of archive bundles, you can simply copy the appropriate archive bundle from the old to the new machine.
  • Or, to make it even more simple, just copy the entire archive folder with all archive bundles from the old to the new machine.
  • The location where Xcode stores archive bundles is visible in the Xcode preferences on the "Locations" tab. The default for recent Xcode versions (Xcode 4 and 5) is this folder:

~/Library/Developer/Xcode/Archives

• To conclude this section, I advise to always make a backup of the archive folder when a new archive bundle is created. Strictly speaking, archive bundles may not be as valuable as the source code, but they are certainly worth preserving if you intend to support your users in the future by analyzing crash logs for older versions of your application.

How to symbolicate

The simplest way how to symbolicate a crash log is this procedure:

1. Launch Xcode
2. Open the Organizer window
3. Switch to the "Devices" tab and select the "Device Logs" section
4. Drag the crash log (must have the extension .crash) from the Finder into the list of device logs
5. The crash log should now appear as a new entry in the list of device logs. Select the crash log entry and you will see its raw content in the right-hand pane.
6. The symbolicating process takes a moment, so just wait a second or two until the right-hand pane updates with the correct symbol. If this does not happen, then your environment is somehow set up incorrectly and you need to troubleshoot.
7. If symbolicating worked correctly, you may wish to save the symbolicated crash log somewhere. Drag the crash log entry from Xcode's Organizer window to some location in the Finder.

Symbolicating can also be done manually on the command line. Read on for more details.

The symbolicatecrash script

Symbolicating is done by a helper Perl script that Xcode runs in the background for you. If you know where the script is you can also run it manually on the command line. The name of the script is

symbolicatecrash

The script is located somewhere inside an Xcode application bundle. For instance, you can use xcode-select to look for the script inside the default Xcode application bundle like this:

find "$(xcode-select -p)" -name symbolicatecrash

Run the script with the -h parameter to find about its command line parameters:

$(find "$(xcode-select -p)" -name symbolicatecrash) -h

I find the -v parameter useful for troubleshooting when symbolicating fails in Xcode's Organizer window. To actually perform symbolicating, however, the environment variable DEVELOPER_DIR must be set. This is the complete command line that attempts symbolicating and writes debugging output into a log file:

DEVELOPER_DIR=$(xcode-select -p) $(find "$(xcode-select -p)" -name symbolicatecrash) -v foo.crash >foo.crash.symbolicated 2>symbolicatecrash.log

Spotlight indexing of .dSYM bundles

The symbolicatecrash script tries to find .dSYM bundles by querying the Spotlight index. The query is done with the mdfind command line utility. You can run the query yourself like this:

mdfind "com_apple_xcode_dsym_uuids == 34405285-2C3C-3F5F-96B2-173728178EEB"

To see all .dSYM bundles that have been indexed, run this query:

mdfind "com_apple_xcode_dsym_uuids == *"

How does symbolicatecrash obtain the UUID to search the Spotlight index? Simple: It takes them from the raw crash log file. Open this file in any text editor and, towards the end, look for a section heading "Binary Images". Entries in this section look like this:

0x93000 -   0x21cfff +Little Go armv7  <344052852c3c3f5f96b2173728178eeb> /Users/USER/Little Go.app/Little Go

Notice the series of hex characters "344052852c3c3f5f96b2173728178eeb" - this is the UUID as an unformatted lower-case string. To convert the string into the canonical UUID format that is required to search the Spotlight index:

echo 344052852c3c3f5f96b2173728178eeb | perl -e 'foreach $uuid (<STDIN>) { chomp($uuid); $uuid = uc($uuid); $uuid =~ /(.{8})(.{4})(.{4})(.{4})(.{12})/; $uuid = "$1-$2-$3-$4-$5"; print "$uuid\n"; }'

Presence of iOS SDKs

In order to be able to symbolicate invocations of system functions, the symbolicatecrash script requires that the appropriate iOS SDKs are installed. In the crash log file, the following line identifies the SDK that must be installed:

OS Version:      iPhone OS 6.1.4 (10B350)

With the information from this line, symbolicatecrash attempts to locate a matching SDK. First it creates a VERSION_PATTERN that matches any one of the following folders:

• 6.1.4 (10B350)
• 6.1.4
• 10B350

Then it looks for the SDK in the following folders:

• /System/Library/Developer/Xcode/iOS DeviceSupport/VERSION_PATTERN/Symbols*
• /Library/Developer/Xcode/iOS DeviceSupport/VERSION_PATTERN/Symbols*
• ~/Library/Developer/Xcode/iOS DeviceSupport/VERSION_PATTERN/Symbols*
  • Usually these folders in your user home directory are populated when you attach an iOS device to the machine and Xcode extracts debug symbols from the device
• Inside all Xcode application bundles that can be found on the system: /path/to/Xcode.app/Contents/Developer/Platforms/*.platform/DeviceSupport/<VERSION_PATTERN>/Symbols*
  • Excluded are platform folders that contain the word "Simulator"
  • Xcode application bundles are found via Spotlight index with this command: mdfind "kMDItemCFBundleIdentifier == 'com.apple.dt.Xcode' || kMDItemCFBundleIdentifier == 'com.apple.Xcode'"

dwarfdump

The dwarfdump command line utility provides information about a .dSYM bundle. For instance, get the UUID of a bundle with the --uuid parameter. This example prints the UUIDs of all .dSYM sub-bundles in all of your archive bundles:

find ~/Library/Developer/Xcode/Archives -name \*.dSYM -print0 | xargs -0 dwarfdump --uuid

To symbolicate just a single line from the crash log, such as this one

5   Little Go                           0x000e8967 0x93000 + 350567

you might use the following invocation of dwarfdump:

dwarfdump --arch armv7 /path/to/xcarchive/dSYMs/Little\ Go.app.dSYM --lookup 0x000e8967

Hacks

To force symbolicatecrash to use a particular .dSYM bundle:

• Run dwarfdump --uuid to find out the UUID of the bundle that you want to use
• Convert the UUID to an unformatted string of lowercase hex characters. For instance, "34405285-2C3C-3F5F-96B2-173728178EEB" becomes "344052852c3c3f5f96b2173728178eeb".
• Replace the original UUID in the crash log in the "Binary Images" section with the lower-case string
• Run symbolicatecrash
• This hack could be useful if you have lost your original .dSYM bundle, but you were able to re-create an identical bundle from the source code that matches the version of your application that produced the crash log.
• I have never tested this hack, but I guess that for it to actually work you probably also need the identical Xcode version that you used to build the application that produced the crash log (otherwise the information in the crash log might not match the content of the .dSYM bundle).

Troubleshooting

Troubleshooting in case Spotlight is not indexing your .dSYM bundles:

• Troubleshooting Spotlight Importers is a generally useful document from Apple
• Run mdimport on the folder that contains your .xcarchive bundles (typically ~/Library/Developer/Xcode/Archives)
• If this does not help, run mdimport -L and check that the list contains the Spotlight importer uuid.importer from inside an Xcode application bundle (e.g. /Applications/Xcode-5.0.2.app/Contents/Library/Spotlight/uuid.mdimporter).
• If uuid.importer is not listed, then you might need to reboot your machine, or at least logout/login with your dev user. This happened to me once when I installed Xcode with a privileged admin user, but then never logged out with the non-privileged dev user that was logged in at the same time when Xcode was installed. As a consequence, the list of Spotlight importers of the non-privileged dev user was never updated to include the newly installed uuid.importer.
• If uuid.importer is listed, then the location of the .xcarchive bundles is excluded from Spotlight indexing for some reason. For instance, in Mac OS X 10.9 I have found that ~/Library/Developer/Xcode/Archives (or, for that matter, anything inside ~/Library) is not indexed by Spotlight - something that must have changed from earlier releases of Mac OS X.
• Finally, to use mdimport with a specific Spotlight importer, run this command

mdimport -g /Applications/Xcode-5.0.2.app/Contents/Library/Spotlight/uuid.mdimporter ~

Code signing

Overview

This section is mostly a condensed form of Apple's official documentation, the "Code Signing Guide" .

Code signing is all about creating digital signatures with public key cryptography, so nothing miraculous. Instead of signing an email or a regular document (e.g. a PDF or MS Word file), with code signing we create a digital signature for a blob of compiled source code (e.g. an object file, a mach-o binary, etc.).

To recap, the workflow to create a digital signature looks like this:

• Take a piece of data (D1) that should be signed as the input
• Use a cryptographic hash function (F) to create a message digest (M1) from the input data
• Use the private key of the signing party to encrypt the message digest
• The encrypted message digest is the digital signature (S)

And this is the workflow to verify the digital signature:

• Take the piece of data (D2), the digital signature (S), plus the certificate of the signing party as the input
• Use the public key embedded within the certificate to decrypt the digital signature (S) and thus receive the original message digest (M1) computed during signing
• Use the same cryptographic hash function (F) that was used during signing to create a message digest (M2) from the piece data (D2)
• Compare M1 and M2. If they are the same, then D2 must be the same as D1

Seals

The overview simplified things a bit. In reality, various parts of an application may be signed together, not just a single piece. Things that may be signed are the app identifier, the Info.plist, the main executable, the resource files, etc. A separate message digest is created for each part of an application that should be signed. All message digests are then collected into a so-called seal, and it's that seal for which the signature is created.

Multiple signatures vs. seals

Apple's "Code Signing Guide" is not too clear when exactly seals are used. One part of the guide mentions that things may also be separately signed:

• If the executable is "universal", i.e. it contains code for several architectures, then each "slice" (= piece of code for one architecture) is signed separately
• Parts of an application (e.g. an Info.plist) are signed separately. These signatures are stored in a file called _CodeSignature/CodeResources within the bundle. There is no explicit statement which parts of the application are thus signed.

Code signing identity

A signing identity consists of

• A private key
• A digital certificate

Requirements for the certificate:

• It must contain the public key that corresponds to the private key
• It must have a usage extension that enables it to be used for code signing

A code signing identity can be created without involving Apple at all. In this case, systems that want to install/execute code signed with that identity must somehow be instructed to trust the identity. This might happen by establishing a trust chain from the certificate of the signing identity up to the root certificate of a CA (certificate authority). Or it might happen simply by forcing the system to trust the self-signed certificate of the signing identity.

Code signing identities provided by Apple

Alternatively, it is possible to obtain the following types of certificates from Apple by submitting a certificate signing request (CSR) to Apple's developer portal. The types of certificates that you can get are different, depending on whether you develop for Mac OS X or for iOS.

Mac OS X:

• Developer ID certificate. Such a certificate can be used to sign code that is publicly distributed, e.g. via a download from a website.
• Distribution certificate. Such a certificate can be used to sign code that is published via the Mac App Store.

iOS:

• Developer certificate. Such a certificate can be used to sign code that will be installed on a device during development, e.g. in order to launch a debugging session.
• Distribution (or production) certificate. Such a certificate can be used to sign code that will be distributed to tester devices during beta testing, or that will be distributed to end users via the iOS App Store.

In all cases the certificate you receive is signed by Apple, albeit in different ways for the various types of certificates. TODO: Provide examples to demonstrate the differences.

codesign

The utility

codesign

is used to sign code.

TODO: Examples

Xcode's use of codesign

TODO: Details about the code signing options in an Xcode project file, especially when the option CODE_SIGN_IDENTITY is set to an empty string, which causes Xcode to use an automatically determined identity.

Entitlements

TODO: What EXACTLY are entitlements, and what is their role in the code signing process

Provisioning profiles

TODO: What EXACTLY are provisioning profiles, and what is their role in the code signing process

spctl

The utility

spctl

is used to manipulate the so-called "security assessment policy subsystem" of Mac OS X. The policies in this subsystem determine whether Mac OS X allows the installation, execution, and other operations on files on the system. In other words, spctl is not involved when the code is signed, but when code signatures are verified.

It is, however, possible to test whether the result of the code signing process is correct. The --assess program option is used for that. For more details, refer to Apple's "Code Signing Guide" and to the man page of spctl.