Tensors are ubiquitous in science and engineering and tensor factorization approaches have become important tools. This paper explores the use of Bayesian modeling in the context of tensor factorization and presents a probabilistic extension of the so-called Block-Term Decomposition (BTD) model and show how it can interpolate between two common decomposition models - the Canonical Polyadic Decomposition (CPD) and the Tucker decomposition. This probabilistic extension is obtained by applying Bayesian inference to the BTD model, allowing for uncertainty quantification, robustness to corruption by noise and model miss-specification. The novelty of this model is its applicability to N^th-order tensors, incorporating mode specific orthogonality within each block, and priors that penalizing complexity of the core arrays. On synthetic and two real datasets, we highlight the benefits of probabilistic tensor factorization considering the BTD, demonstrating that the probabilistic BTD can successfully quantify multi-linear structures and is robust to noise.