Here's the source code of the script that limits memory and time consumption of a "black-box" process in Linux. Its implementation details are thoroughly described in the relevant blog post.
The bleeding-edge latest version (and the contributions made by others but not yet merged) may be downloaded from the GutHub page for timeout script.
See README with usage information and examples on GitHub as well.
The script is licensed under Apache-2.0, and was developed mainly during my day job in ISPRAS while working on Linux Driver Verification project. I designed and implemented the most part of the script, several contributions were made from Alexander Strakh.