Last month, I was invited to Melbourne for a pharmacometrics conference: Population Approach Group of Australia & New Zealand (PAGANZ) meeting. It was great getting to know some more pharmacometricians and really digging into the problems they face, specifically in statistical inference as applied to PK/PD models.
It was a busy trip with talks / workshops lined up on all three days:
One day Stan course as part of Population Analysis Work Shops (PAWS) taught by myself and Sam Brilleman.
ISoP Lecture at PAGANZ titled “Stan Meets Pharmacology.”
Melbourne Stan Meetup talk titled “Understanding lp__: proportionality constants and (automatic) transforms.”
I wanted to fill in some of the motivation, especially for the pace of the short course.
We were invited to give a one day Stan course as part of the intermediate workshop for pharmacologists. I’ve given this type of course about a dozen times now and it’s evolved each time I’ve taught it.
First, it’s really ambitious to try to learn a programming language in one day. It’s even harder when the output isn’t deterministic and there are statistical concepts needed that may not be used day-to-day. My goal for the course was to provide enough information where the students could continue to learn after the course.
At a high level, the course covered:
- a (very) quick recap of Bayesian statistics
- why we need Markov Chain Monte Carlo
- the basics of the Stan language: syntax and semantics
- expressing a multi-compartment PK/PD model specified as an ODE in Stan
While that describes the content of the course, I used the time as an opportunity for the students to learn things that aren’t in the Stan Manual or are easier to learn in person. Here are some of the things I focused on:
Stan’s variable types. This is particularly important for pharmacometricians that want to specify ODEs. Stan is a strong, statically typed language, unlike R or Python. Learning to navigate the Stan types is essential to writing models with ODEs. Although it’s written out very well in the Stan manual, lots of users are still confused. There are a lot of exercises in the slides for the students to get comfortable with the types.
Error handling. One of the most frustrating things about learning a new skill is failing, not understanding why it’s failed, and not understanding how to address it. Stan’s error messages are actually pretty good. But these error messages look completely different than error messages in R or Python. I demonstrate ways to generate different error messages, how to read the messages, and how to interpret them. The students also work through common errors in the hopes that after the course most errors (even ones that weren’t seen in the course) aren’t hurdles.
Practice. There’s no substitute for putting in the hours. During the day, there are numerous exercises that vary in difficulty that flex things learned.
How to navigate the Stan manual. Or at least how I navigate the Stan manual. It’s wonderfully written, but it’s approaching 700 pages now. I read it cover to cover when it was under 200 pages. There are some tricks to navigating the current pdf manual.
The patience required to work with Stan. Stan is not only a compiled language, the C++ compilation takes a long time. On my Mac, a model takes about 30 seconds to compile. On the Windows machines the students were working with, it was about 2 minutes. This really feels like an eternity compared to running a single line in R or Python. Explanation of the steps help a bit. And so does the demonstration.
Where to get additional help. I really try to tell the students that they’re not alone in learning the language. Unfortunately, there are no intro Stan programming books yet, but the resources are out there. And for anyone reading this, if you run into trouble, reach out on the Stan forums.
After these courses, I’m usually asked why I don’t provide working examples to all the students. I’ve done that once and I found that to be one of the biggest impediments to gaining competency in Stan. There is definitely a different type of course that could be taught that is aimed entirely at interpreting Bayesian inference in Stan. This course was aimed at students that want to write statistical models.
Talk: “Stan Meets Pharamcology”
This was a fun talk for me. I’ve been working on Stan for the last 7 years while keeping track of algorithmic and implementation developments in computational statistics and machine learning for the last 10. It was a chance for me to talk about how Stan has transformed statistical computing and some of the pharmacology applications Generable has been involved with recently.
I completely mistimed the talk and ended early. Fortunately, the talk sparked a lot of discussions about Bayesian inference and how it applies to PK/PD modeling and we were able to have a conversation about some of the finer points.
Talk: “Understanding lp__”
This was for the Melbourne Stan meetup. I had wanted to discuss how
lp__ is computed in Stan for a while. The exact computation is covered in the manual, but for many users, it’s still a mystery. We spent the hour understanding what
lp__ is computing, writing the expression on paper for a simple model, writing an R program to compute it, and verifying the result through Stan. I’ll have to give this talk again soon.
The PAGANZ community was great and hopefully I’ll have a chance to collaborate with a lot of the members soon. There are a few people I want to thank for making this trip happen:
- Carl Kirkpatrick for the invitation to teach the course and present at PAGANZ and for arranging everything!
- Sam Brilleman for helping with the course
- Steve Duffull for his hospitality and lively discussion about Bayesian inference
- Martin Ingram and Andy Kitchen for hosting the Melbourne Meetup