An encryption scheme is only as strong as its weakest link. In academic terms, only the algorithm really matters. In the real world however, implementation is as important.
The human element is an element that has to be considered. Rubber hose cryptanalysis is a tongue and cheek way of acknowledging that. It also matters since some algorithms are better at assisting here. E.g. 1 time key Vs passwords.