Tom,
Latex and Silicone... Somewhat the same thing is a pretty vague question but I will try to answer in accordance to what you are using them for. You are probably using the brushable latex, which is good for making molds of small parts. Latex does not have the structual integrity or the logevity that Room Temperature Vulcanizing Silicone (RTV) has. In other words, quite a few factors come in to play depending on the size of the piece being molded, how many castings you expect to get from your molds and the detail of the master itself. A properly prepared moldbox with the master properly imbedded (see my article)
https://armorama.kitmaker.net//features/469 is hard to beat when pourable RTV is used. Because it is purable, it finds every detail allowing you to make perfect reproductions of your master.
Plaster of Paris, because of it's lack of flexibility, are usually good for one time molds. It's advantage is that it is inexpensive, but if you have a fine detailed piece, I caution against POP as it's ability tp pick up super fine detail is one of it's many misgivings.