Interpreting MacOS Crash Reports: Diagnosing Issues in FileMaker Server
After the Windows crash report from earlier today where we couldn’t do much, here is an example for one where we could help on MacOS.
When you find a crash report in /Library/Logs/DiagnosticReports folder (for server software) or ~/Library/Logs/DiagnosticReports (for user software), you can try to read where it crashed and maybe avoid that code or fix the data processed by the code to avoid the problem.
The crash report we check today starts with this basic info:
Process: fmscwpc [19163]Path: /Volumes/*/fmscwpcIdentifier: fmscwpcVersion: 0Code Type: X86-64 (Native)Parent Process: fmserver_helperd [19089]Responsible: fmscwpc [19163]User ID: 503
Date/Time: 2019-09-26 13 : 32:54.009 +0200 OSVersion: Mac OS
As you can read, we have the process name fmscwpc, which is FileMaker Server Custom Web Publishing Client as far as I know. This is the process of FileMaker Server handling requests for WebDirect. It runs here on MacOS 10.13 and you see the timestamp when it happens. When you can reproduce the issue, you may get a few crash reports, so having times may help to distinguish them. Next section shows details on the problem:
Time Awake Since Boot: 10000 seconds
System Integrity Protection: enabled
Crashed Thread: 103
Exception Type: EXC_BAD_ACCESS (SIGSEGV)Exception Codes: KERN_INVALID_ADDRESS at 0x0000000000000000Exception Note: EXC_CORPSE_NOTIFY
Termination Signal: Segmentation fault: 11Termination Reason: Namespace SIGNAL, Code. 0xbTerminating Process : exc handler [0]
VM Regions Near 0:–>__TEXT 000000010a8f5000-000000010aa92000 [ 1652K] rx/rwx SM=COW /Users/USER/*
We read here that crash 103 crashed with a bad memory access error. Some memory was accessed, eg a buffer overrun where you only have 100 elements, but you try to access the 200th element and nobody checked bounds.
Scrolling down through over 100 threads, we find the thread 103 and here is the call stack:
Thread 103 Crashed:0 fmscwpc 0x000000010a9a841e iwp::IWPLayoutParts::GetDataForSpecifiedPopover(int, unsigned long long, std::__1::map, std::__1::allocator> >&, std::__1::map, std::__1::allocator> >&, bool, short, int, int) + 7261 fmscwpc 0x000000010a9ae642 iwp::IWPLayoutAccessor::GetPopoverData(int, unsigned long long, int, short, int, bool, std::__1::map, std::__1::allocator> >&, std::__1::map, std::__1::allocator> >&) const + 642 fmscwpc 0x000000010a976ae5 iwp::IWPServiceImpl::getDataForCurrentPopover(layout::LayoutDataResult&, context::Context const&, int, int, int) + 2173 fmscwpc 0x000000010a951835 i wp::IWPServiceProcessor::process_getDataForCurrentPopover(int, apache::thrift::protocol::TProtocol*, apache::thrift::protocol::TProtocol*, void*) + 4934 fmscwpc 0x000000010a94be6a iwp::IWPServiceProcessor::process_fn(apache::thrift::protocol::TProtocol*, apache::thrift::protocol::TProtocol*, std::__1::basic_string, std::__1::allocator>&, int, void*) + 1065 fmscwpc 0x000000010a94bc3a iwp::IWPServiceProcessor::process(boost::shared_ptr, boost::shared_ptr, void*) + 1146 libthriftnb-0.9.2.dylib 0x000000010acafbc3 apache::thrift::server::TNonblockingServer::TConnection::Task::run() + 2437 libthrift-0.9.2.dylib 0x000000010abd34ce apache:: thrift::concurrency::ThreadManager::Task::run() + 308 libthrift-0.9.2.dylib 0x000000010abd2d55 apache::thrift::concurrency::ThreadManager::Worker::run() + 5979 libthrift-0.9 .2.dylib 0x000000010ac10872 apache::thrift::concurrency::PthreadThread::threadMain(void*) + 17810 libsystem_pthread.dylib 0x00007fff66025661 _pthread_body + 34011 libsystem_pthread.dylib 0x00007fff6602550d _pthread_start + 377.12 libsystem_pthread.dylib 0x00007fff66024bf9 thread_start + 13
You read it from bottom up, so the thread starts and runs thread manager functions in apache framework to do tasks. One of the tasks is in IWPServiceProcessor class to process some requests. It calls the getDataForCurrentPopover function here to get data and the inner call to GetDataForSpecifiedPopover causes the problem by accessing some data, which is not there. So this problem seems to be related to loading a popover.
For a crash report always check further down the versions of the loaded libraries. When I see Adobe PDF plugins, I recommend to delete them and if I spot old plugins, I also recommend to update those. The last section is the memory summary. If the application uses a lot of memory, it may have been crashed by running out of available memory. But our example crash report just shows 1.2 GB, which seems to be a reasonable size for this process.
After talking back to client, they could see the crash happens when the layout with the popover is used. When the popover was removed and later replaced with a new one, the problem went away. The client is happy and I hope he also reported it to Claris Inc., so it can be fixed in a future version of FileMaker Server, if it is not fixed already and this server is not yet updated. 15 10 19 – 09:20
