A 3D Steganalytic Algorithm and Steganalysis-Resistant Watermarking

We propose a simple yet efficient steganalytic algorithm for watermarks embedded by two state-of-the-art 3D watermarking algorithms by Cho et al. The main observation is that while in a clean model the means/variances of Cho et al.’s normalized histogram bins are expected to follow a Gaussian distribution, in a marked model their distribution will be bimodal. The proposed algorithm estimates the number of bins through an exhaustive search and then the presence of a watermark is decided by a tailor made normality test or a <inline-formula> <tex-math notation="LaTeX">$t$</tex-math><alternatives><inline-graphic xlink:href="yang-ieq1-2525771.gif"/> </alternatives></inline-formula>-test. We also propose a modification of Cho et al.’s watermarking algorithms with the watermark embedded by changing the histogram of the radial coordinates of the vertices. Rather than targeting a continuous statistics such as the mean or variance of the values in a bin, the proposed watermarking modifies a discrete statistic, which here is the height of the histogram bin, to achieve watermark embedding. Experimental results demonstrate that the modified algorithm offers not only better resistance against the steganalytic attack we developed, but also an improved robustness/capacity trade-off.


INTRODUCTION
D Igital watermarking is the process of embedding dig- ital signals into digital media such as images, audio, video, or 3D models.In a relationship analogous to that between cryptography and cryptanalysis, as a counterpart to watermarking, steganalysis aims at the detection of watermarks hidden into digital signals.
Thus far, numerous excellent watermarking techniques for inserting watermarks into 3D models have been proposed [1], [2], [3], [4], [5].For a more exhaustive comparison of 3D watermarking methods, we refer the reader to the survey in [6].These algorithms are primarily concerned with the robustness of the watermark, targeting applications such as proof of ownership and copy control, while the undetectability of the embedded watermark does not seem to be a major concern in their evaluation.That also means that 3D model steganalysis is an underdeveloped area and, to the best of our knowledge, there are only a few works [7], [8] in the literature proposing steganalytic algorithms specifically targeting 3D model watermarking.In contrast, the domain of 2D image watermarking has been greatly influenced over the years by the various proposed steganalytic methods [9], [10], [11], [12], [13].
The thesis motivating this paper is that 3D steganalysis can have a similar positive influence on the development of the field of 3D watermarking in at least two ways.Firstly, by opening new applications domains where the understanding of the anti-steganalytic properties of the watermark are either absolutely essential, e.g., covert communication, or at least extremely important, e.g., proof of ownership and copy control through invisible watermarking.Secondly, by motivating the development of improved watermarking algorithms, which are quite likely to have superior properties not only regarding the undetectability of the watermark but also its robustness.

Overview
Steganalytic approaches are classified into two categories: specific and universal.The former detects the presence of a message embedded by particular watermarking algorithms, while the latter aims at message detection regardless of the embedding algorithms used.In this paper we propose a specific steganalytic algorithm for determining the presence of a watermark hidden by Cho et al.'s mean and variance based algorithms [14], which are major 3D watermarking techniques with considerable impact on subsequent research [4], [5].The proposed algorithm exploits the alteration of the model's natural statistics caused by Cho et al.'s watermark insertion method.More specifically, watermarking with Cho et al.'s method makes the distribution of the means and variances of the normalized histogram bins bimodal, while it is expected to be Gaussian before watermarking.
We also propose a blind 3D watermarking algorithm with improved undetectability and robustness performance over Cho et al.'s.The new algorithm embeds the watermark into the histogram of the radial coordinates of the mesh vertices, as Cho et al.'s does.The main difference is that instead of embedding each watermark bit inside a continuous statistics of the model, e.g., the mean or the variance of a normalized histogram bin, we embed it inside a discrete statistic, that is, the difference in the number of elements of two adjacent bins.Experimental results show that the proposed discrete statistic offers improved performance against both the proposed steganalytic attack and standard watermark removal attacks.
The main contributions of the paper are: • A steganalytic algorithm specifically designed against the two watermarking algorithms proposed by Cho et al. [14], which outperforms the universal steganalysis proposed in [8].
• A new 3D watermarking algorithm which is more robust than two state-of-the-art techniques [7], [14] in terms of undetectability against the proposed steganalytic attack, offers a better distortion/capacity trade-off and it is robust against standard watermark removal attacks such as smoothing, noise insertion, subdivision and simplification.
These two contributions introduce to the field of 3D watermarking a new approach into the development of new algorithms, which is analogous to the standard paradigm in image watermarking.In that paradigm, watermarking algorithms are developed through a competition between steganographers and steganalysts.
The main limitation of the proposed steganalytic method is its narrow application domain, that is, it is tailored to work against Cho et al.'s watermarking algorithms.Nevertheless, we reasonably expect that the method can also be used against any other method embedding watermarks by changing the means or variances of specific attributes of the model.

Related Work
Similarly to image watermarking, 3D watermarking techniques are broadly classified into two categories: spatialbased and transform-based.In the spatial domain, Yeo et al. [15] propose a fragile watermarking method which perturbs a vertex ensuring that predefined hash functions have the same value on it.One of its drawbacks is the causality problem, due to its heavy dependence on the order of traversal of vertices.Lin et al. [2] address this issue using vertexorder-independent hash functions.To increase robustness, Yu et al. [16] and Cho et al. [14], instead of inserting the watermark into a single vertex, embed each watermark bit into a group of vertices.Bors [17] uses a neighborhood localized measure to select the vertices that give small embedding distortion and watermark these vertices by local perturbations.Aiming at robustness against mesh editing or pose deformation, Yang et al. [4] propose a Laplacian coordinates based algorithm where the watermark bits are hidden by altering the histogram of the lengths of the Laplacian vectors.
Frequency analysis based algorithms can achieve excellent results on both watermark robustness and imperceptibility.By using the spectral analysis by Karni et al. [18], Ohbuchi et al. [19] propose an algorithm embedding the watermark into the low frequencies of Karni et al.'s decomposition.This method is non-blind, thus requiring the availability of the original model during watermark extraction.Praun et al. [1] propose a robust, non-blind watermarking method using an edge collapse based multiresolution decomposition.Kanai et al. [20] propose a nonblind method for semi-regular meshes based on the modification of wavelet coefficients, while Uccheddu et al. [21] extend this approach to be a blind one.
Closely related to watermarking are steganographic methods.While the distinction between watermarking and steganography is not sharp, the term steganography is largely used when we favor large embedding capacity, usually at the expense of robustness, while watermarking schemes are evaluated principally on their robustness.Cayre et al. [22] introduce a steganographic algorithm treating each triangle as a two-state object, depending on the position of the projection of a vertex onto its opposite edge.The maximum capacity this method can achieve is 1 bit per vertex.Inspired by this idea, Wang et al. [23] increase the embedding capacity while minimizing the embedding distortion via using a multi-level hiding procedure.Subsequently, they [24] extend their prior work [23], taking into account texture information during embedding.Two steganographic methods include Chao et al. [25] and Yang et al. [26].All the above mentioned steganographic approaches can achieve high capacity and low distortion, but the main limitation is their weak robustness, that is, they cannot withstand malicious attacks aimed at destroying the embedded message.
The area of steganalysis has been primarily developed on images.Fridrich et al. [27] and Ker [28] propose methods specific for the detection of LSB replacement.Farid [29] proposes a universal approach which uses a wavelet-like decomposition to build higher-order statistical models of natural images.Other universal steganalytic approaches for images include Xuan et al. [30], Wang et al. [12] and Lie et al. [10].
Compared to image steganalysis, 3D steganalysis is a more challenging task since the typical 3D models are more complex objects, having arbitrary topology and irregularly sampled geometry.As such, 3D steganalytic techniques are seriously underdeveloped and considerable research effort is required before approaching a level of maturity similar to that of image steganalysis by solving the various open problems in the field.Farid's method for image steganalysis [29] has been extended in [8] to 3D meshes.In Yang et al. [7], a specific 3D steganalysis is proposed based on the observation that the natural statistics of the 3D model are disturbed by Cho et al.'s mean based embedding.The current paper is an extended version of [7] and includes a new watermarking scheme with smaller distortion compared to [7], [14] and better steganalytic properties that make it practically undetectable by the developed steganalytic attack.The embedding algorithm is based on histogram shape modification, similarly to some existing image watermarking algorithms [31], [32].

STEGANALYTIC ALGORITHM
In this section, we first briefly describe the two watermarking algorithms by Cho et al. [14] and then present in detail the proposed steganalytic attack against these two methods.

Cho et al.'s Watermarking Algorithms
Both mean-based and variance-based variants of Cho et al.'s method embed the message by perturbations of the radial coordinates of the vertices in a spherical coordinate system.Mean-based watermarking: First, the radial coordinates of all the vertices are computed with respect to the barycenter   of the set of vertices of the 3D mesh model.Then, a K bin histogram of the radial coordinates is constructed and the elements in each histogram bin are normalized in the interval [0, 1].Let Bk = {ρ k,j : j = 1, 2, 3, ...} denote the k-th (1 ≤ k ≤ K) bin of the normalized radial coordinates ρk,j .They embed a -1 (+1) watermark bit in that bin by perturbing the mesh vertices to make the mean value m k smaller (respectively, greater) than 1/3, assuming that the mean of the elements ρk,j is zero.

The Proposed Steganalytic Algorithm
Our steganalytic algorithm is based on the observation that the mean-based and variance-based embeddings of Cho et al.'s algorithm result in a 2-clustering of the set of the mean values M = {m k : 1 ≤ k ≤ K} or variances V = {v k : 1 ≤ k ≤ K}, respectively, (see Fig. 1).For brevity, we will denote either of these two sets as M/V when the distinction is not necessary.

Estimation of K:
The main challenge towards a fully automatic steganalytic attack against Cho et al.'s methods is finding the number K of histogram bins that was used in the phase of watermark embedding.If a wrong value of K is picked for the reconstruction of the histogram bins for the purposes of steganalysis, then the distribution of M/V will be indistinguishable between watermarked and clean models.That is, with an incorrect K, the two clusters C and C of M/V will not be well separated even for watermarked models (see Fig. 1).Consequently, the first step of the proposed steganalytic algorithm finds an estimate of K obtained by an exhaustive search through all possible values.Using a standard clustering algorithm based on Expectation Minimization (EM), for each K * we classify the elements of M/V into two clusters C and C by fitting a mixture of two Gaussians [33].We estimate the degree of separation between C and C by the Bhattacharyya distance D K * [34] between the two Gaussians of the mixture model and estimate K by K min and K max define the range of K we would like to consider; here we fix K min = 30 and K max = 500.in Eq. 5 is a user-specified constant preventing the selection of a very uneven clustering with respect to the size of the two clusters.The justification of the constraint is the assumption that the watermark bits follow a uniform random distribution [35], and hence we expect |C| ≈ | C|.Without the constraint, the distance maximization in Eq. 4 might return as optimal a clustering consisting of a small cluster containing a few outliers and a large cluster with all the other values.Given the estimate of K, we employ a tailor made normality test or a standard t-test to make a steganalysis decision, that is, to decide whether M/V are samples from a single Gaussian, in which case we have a clean model, or they come from a mixture of two Gaussians, in which case we have a watermarked model.
Normality Test: Even though standard normality tests exist, here we use a test specifically designed for the extreme cases we deal with.Indeed, since K is selected for making the distribution of M/V as bimodal as possible, a less sharp test may reject the normality assumption even in clean meshes.
We make use of the Q-Q plots, which plot the quantiles of two distributions against each other.The first distribution is the empirical distribution of M/V and the second is the standardized normal distribution.If two distributions are linearly related, here if M/V are linearly related to the normal distribution, the points in the Q-Q plot are nicely modeled by the reference line [36] where µ and σ are the mean and the standard deviation of M/V.
We check if the reference line is a good model of the points of the Q-Q plot by comparing it with the least square linear fit of these points.That is, we compute the angle of these two lines where abs (x) stands for the absolute value of x; s denotes the slope of the least square linear fit, and compare it against a threshold θ T .If θ > θ T , we assume that the reference line is not a good model of the Q-Q plot, hence M/V is not Gaussian and the mesh is marked.Fig. 2 shows the Q-Q plots, the reference line and the least square linear fit for M of the clean and a marked Rabbit model.To reduce the impact of outliers in M/V, we compute the least square linear fit from the points in the range [-0.5, 0.5] of the normal distribution in the Q-Q plot, the gray area in Fig. 2.

t-test:
In contrast to the normality test which assumes that the set M/V of a clean model follows a single Gaussian distribution, t-test is based on the hypothesis that the two clusters C and C are independent random samples from Gaussian distributions with equal means.The main observation is that while C and C of unmarked models should have certain overlap, C and C of marked models are expected to be clearly separated.Consequently, the rejection of the hypothesis of equal means would imply a marked model.
Thus, in this second variant of the steganalytic algorithm, the steganalytic decision is based on a t-test with significance level α, with null hypothesis that the data in C and C of M/V are independent random samples from normal distributions with equal means and unknown variances, against the alternative that the means are not equal.

MODIFIED WATERMARKING ALGORITHM
In a bid to improve the anti-steganalytic properties of Cho et al.'s algorithms [14], we developed a watermarking algo-rithm utilizing discrete rather than continuous statistics of the histogram of the radial coordinates.Each watermark bit w i ∈ {−1, +1} is embedded into a pair of bins (B k , B k+1 ) and its value depends on whether the bin Bk is shorter or taller than Bk+1 , that is, on the sign of the difference of the number of elements in the two bins | Bk | − | Bk+1 |.The embedding process manipulates the difference | Bk | − | Bk+1 | by considering the triplet of neighboring bins (B k , B k+1 , B k+2 ) and possibly transferring among them some elements, carefully selected to minimize embedding distortion.
In a second modification to Cho et al.'s algorithms, the origin O of the spherical coordinate system is not the barycenter of the vertex set, but it is instead a point chosen to improve the trade-off between embedding distortion and robustness.
Compared to the watermarking method proposed in [7], the two main improvements in our current approach are: • Instead of transferring elements between two bins (B k , B k+1 ), the embedding process transfers elements among three bins (B k , B k+1 , B k+2 ).By considering triplets rather than pairs of bins, we are able to select the to-be-modified elements from a larger set, further reducing the embedding distortion.Notice that as the exact amount of embedding distortion depends on a multitude of factors, such as the the value of K and the specific sequence of watermark bits, the use of three-bin watermarking does not always guarantee lower embedding distortion.However, the experiments have shown that, on average, the threebin watermarking offers a significant reduction of the distortion as compared to the two-bin method.
• While [7] uses an empirically found point on the principal axis as the origin O of the spherical coordinate system, the current method finds O by solving an optimization problem, resulting in a more evenly distributed histogram of the radial coordinates.Such optimized histogram is able to carry more watermark bits than the one in [7], since it has more embeddable pairs of bins (see Eq. 11).In addition, embedding over more even histograms can reduce the number of element transfers, reducing embedding distortion.Indeed, when |B k | < |B k+1 | occurs in a less evenly distributed histogram, we have to transfer more elements in order to achieve | Bk | > | Bk+1 | and encode a watermark bit.

Watermark Embedding
The embedding process is described in detail as follows.
Step 1: The first step is to compute the origin of the spherical coordinate system.Let (x i , y i , z i ) be the Cartesian coordinates of the i-th vertex of a polygonal mesh with N vertices.We compute the barycenter (x c , y c , z c ) of the vertex set and obtain its principal axis v after applying Principal Component Analysis (PCA) to it.The origin O = (x, ȳ, z) of the spherical coordinate system is a point on the principal axis, that is, on line passing through (x c , y c , z c ) with the direction v t, which is computed through an exhaustive search, minimizes the total variation in the heights of triplets of neighboring bins [t min , t max ] defines the range we would consider for t and is computed so that the resulting (x, ȳ, z) candidates are within the bounding box of the 3D model; abs (x) denotes the absolute value of x.Unlike [7] which uses an empirically selected point as the origin of the spherical coordinate system, the proposed watermarking method chooses for origin a point that gives a histogram with smooth variation of bin frequencies.As mentioned earlier, the two advantages are: (i) more embeddable triplets of bins and (ii) lower embedding distortion.
Following a standard notation [37], the spherical coordinates of the vertex (x i , y i , z i ) are denoted by Step 2: We build a histogram with where ρ min and ρ max are the minimum and the maximum of P, and is the range size of each bin.We also assume that ρ max ∈ B K .
Step 3: Starting from the second bin B 2 , we arrange adjacent bins into pairs (B 2 , B 3 ), (B 4 , B 5 ), . . ., (B k−2 , B k−1 ) and hide a watermark bit into each embeddable pair of bins.A pair is embeddable if Notice that no watermark bits are carried by the bins B 1 and B K .That means that both end vertices of the projection on the principal axis do not move during embedding, increasing the robustness under blind extraction.If K is odd, the bin pairing process requires to exclude one more bin; here A watermark bit w i is embedded in an embeddable pair (B k , B k+1 ) by transferring some elements among the bins of the triplet (B k , B k+1 , B k+2 ) via increasing or decreasing their values.More specifically, to encode/insert w i = −1, we move elements from the bins B k and B k+2 into B k+1 until, if possible, where Bk is the k-th watermarked histogram bin and n thr ≥ 1 is a user specified integer threshold used to control the robustness/distortion trade-off.Specifically, we increase the values of n left elements ρ i of B k according to and decrease the values of n right elements ρ i of B k+2 according to pushing them into B k+1 .Here, ρ + i and ρ − i are the new radial coordinates and ρ k+1 min and ρ k+1 max are the minimum and the maximum in B k+1 , respectively.The denominator of the fraction in Eq. 13 is an integer, chosen such that ρi is inside the range of the existing elements of B k+1 , that is, ρ k+1 min < ρ + i < ρ k+1 max , and it is as near to ρ k+1 max as possible.Similarly, Eq. 14 makes ρ − i as near to ρ k+1 min as possible.To minimize the embedding distortion, the n left elements ρ i of B k and n right elements ρ i of B k+2 to be moved into B k+1 are chosen as those minimizing the distortion measure Since B k+2 will be used to carry the next watermark bit w i+1 and thus the histogram's even distribution property needs to be maintained, we only consider a subset of B k+2 when moving elements.
, then n left = 0 and n right = 0, meaning no alteration is required.Case 2: Else we repeatedly transfer an element from B k or B k+2 into B k+1 , incrementing n left or n right by one, respectively, and we stop when Eq. 12 is satisfied.Notice that this simple incremental process minimizes Eq. 15 and no global optimization over all the elements of B k and B k+2 is required.
The embedding process for w i = +1 is very similar and the details are omitted.
Step 4: After embedding the watermark bits into the radial coordinates, we convert the spherical coordinates (ρ i , φ i , θ i ) to the Cartesian coordinates (x i , ŷi , ẑi ) according to Eq. 16 where ρi stands for the watermarked ρ i , producing the watermarked 3D model.

Watermark Extraction
The watermark extraction process is straightforward and can be performed with no reference to the original 3D model.Given a marked mesh, we compute the set of watermarked radial coordinates and construct a histogram with K bins B = { Bk : 1 ≤ k ≤ K}.We form the pairs of bins ( B2 , B3 ), ( B4 , B5 ), ( B6 , B7 ), • • • and count the number elements | Bk | in each bin.Finally, the watermark bits ŵi are sequentially extracted from each pair ( Bk , Bk+1 ) by TABLE 1 Comparison between the universal steganalysis in [8] and the proposed specific steganalysis with = 0.15 applied on: Cho's et al.'s methods; their variants with β = 10% of the bins left intact; Yang et al.'s watermarking [7]; the proposed watermarking.The fourth column shows the accuracy in the estimation of K and the fifth the accuracy of the final steganalytic decision.In the fifth column, the left and right subcolumns, where applicable, show the accuracy rates corresponding to the tailor made normality test and the t-test, respectively.

EXPERIMENTAL RESULTS
The performance of the proposed steganlytic algorithm against the original Cho et al.'s methods including their variants with some bins left in purpose unmarked and the improved version proposed here is validated in Sections 4.1 and 4.2, respectively.Note that while many steganalytic approaches [27], [29] have been proposed for digital images, to the best of our knowledge, [8] is the only steganalytic methods for 3D models available in the literature.Consequently, we compare the proposed method against the 3D steganalysis in [8] only.

Steganalytic Algorithm Validation
We validated the steganalytic algorithm on a test set consisting of 445 clean models, mostly from Princeton's University repository [38], and their marked counterparts.As some 3D meshes are unable to carry the watermark for certain values of K, we might have different numbers of marked models for different K's.The results, using a fixed value of = 0.15 in Eq. 5, are summarized in Table .1.The sensitivity of the parameter will be discussed at the end of this subsection, while accuracy rates with treated as a variable are shown in Fig. 4. As a first observation, we notice that the proposed method for estimating the number of bins K achieves high accuracy rates for both of Cho et al.'s algorithms.For example, the accuracy rate for the estimation of K reaches as high as 98.65% for the variance based method when embedding 64 watermark bits into the clean models.Due to the successful estimation of K, the final steganalytic decisions are also highly accurate and in particular, as expected, the proposed specific steganalytic methods outperform the universal algorithm [8].
Normality Test: Fig. 3 plots the detection accuracy with respect to the angle threshold θ T , measured in degrees, for Cho et al.'s methods for watermarks of sizes 64 and 100 bits, respectively.The figure implies that satisfactory steganalysis rate (e.g., ≥ 95%) can be obtained when we set the threshold θ T to any value within the interval [2,3] for the mean based watermarking and any value within [2,13] for the variance based watermarking.
By choosing a value of θ T = 2.8 and θ T = 5.2, the proposed steganalytic method detects the existence of watermark with accuracy rates of 98.52% for Cho et al.'s mean based algorithm and 99.32% for the variance based algorithm when embedding 64 bits, see Table .1.

t-test:
To justify the use of the tailor made normality for the final steganalytic decision, we compare it against the standard t-test.Fig. 3  Fig. 4. The accuracy rate plotted against the parameter , using the tailor made normality test, for 64 (left) and 100 (right) watermark bits.
that when embedding 100 bits we can obtain a detection accuracy above 95%; however, that rate is still inferior the one achieved by the tailor made normality test and, moreover, the highest accuracy rates correspond to extremely small values of the confidence α, raising numerical stability concerns.When embedding 64 bits, the t-test is not able to achieve detection rates above 95% and starts degrading when we use very small values of α.
For a direct comparison of the accuracy rates obtained by the two approaches, in Table 1 we also report the accuracy rates of the t-test for optimal values of α.

Variants of Cho et al.'s:
The high detection rates of the proposed steganalytic algorithms are a direct consequence of the clear separation between the clusters C and C (see Fig. 1) in a marked model, when of course the correct value of K is used.Thus, as a quick fix to the problem of improving the steganalytic properties of Cho et al.'s algorithms, one natural idea is to leave a percentage β of the histogram bins intact, i.e., unmarked, to enervate the degree of separation and hence to frustrate the proposed steganalysis.We refer to the algorithms obtained by this technique as the variants of the originals.Fig. 2 (right) illustrates the Q-Q plot of the mean values of the Rabbit model watermarked by the variant of Cho et al's mean based method with 100 watermark bits and β = 10%.
Fig. 3 shows the accuracy rates obtained on the variants of Cho et al.'s with β = 10%, using exactly the same steganalytic algorithm we applied on the original.We notice that in conjunction with the tailor made normality test the algorithm is very robust in handling the variants, achieving detection rates very similar to those obtained for the original algorithms.On the other hand, we also notice that the range of values of the threshold θ T giving near optimal results has been narrowed.The t-test, again, significantly underperformed the tailor made test.
Parameter Sensitivity: The estimation of K involves the use of a parameter which prevents false choices of K resulting from clusterings consisting of one small cluster of outliers and a much larger cluster with all other values of M/V.The use of that parameter is justified by the assumption that the watermark bits are i.i.d.random variables following the Bernoulli distribution with p = 0.5, that is, 0.5 probability for either a +1 or a -1 bit, which is generally true in practical applications.To test the robustness of the algorithm against the choice of we repeated the tests treating as a variable.The results, obtained with the use of the tailor made normality test, are shown in Fig. 4 and confirm that the proposed steganalysis is largely insensitive to the choice of .

Watermarking Algorithm Validation
We evaluated the proposed watermarking algorithm on a small representative set of well-known 3D models, consisting of the Bunny, Rabbit, Venus, Dragon and the Horse, see Fig. 5.The validation tests include embedding distortion, embedding capacity and watermark robustness.The size of the models and the watermarking parameter used are listed in Table 2.
To measure the amount of distortion caused by the embedding of the watermark we utilized two widely used quantitative measures: the root mean square error (RMSE) with respect to the bounding box diagonal and the MSDM2 distance [39] between the original and the marked models.Notice that the MSDM2 distance correlates better with human vision than the Hausdorff distance.The computations were done with the Metro tool [40] and Mepp 1 , respectively.Regarding the robustness of the watermark, we used the standard measure of the correlation coefficient where w and w are the means of the inserted watermark sequence w and the extracted sequence w , respectively.
Embedding Distortion: Fig. 7 plots the RMSE and MSDM2 distances against the number of histogram bins K.We notice that the values of both error measures are low, showing that the proposed watermarking causes only slight distortions to the original carrier models.We also observe that an increase in the value of K generally leads to a decrease of the embedding distortion.The reason is that by increasing K we decrease the bin width ∆ ρ (see Eq. 10) and hence decrease the modulation of the radii ρ i when they are transferred from one bin to an adjacent.
To gauge the visual significance of the distortion, Fig. 6 shows several marked models for K = 400.Any distortion caused by the watermark insertion is hardly noticeable; however, after zooming in we might be able to observe some artifacts in the smooth areas of the model.Fig. 8 shows back to back close-ups of the clean and the watermarked Bunny and Rabbit.

Embedding Capacity:
The theoretical maximum length of a watermark bit sequence that can be embedded is (K − 2)/2 .However for large values of K this maximum capacity is usually unachievable because some triplets 1. http://liris.cnrs.fr/mepp/Watermark Robustness: The proposed method is robust against distortionless operations such as vertex reordering, translation, rotation, uniform scaling and their combinations, due to the invariance of the histogram of the radii ρ i under such transformations.Watermark robustness was also evaluated against common malicious attacks, such as noise addition, smoothing, quantization, subdivision and quadric edge collapse.We tested against these attacks, fixing K = 400 and varying strength of the attack, following the mesh watermarking benchmark [41].Fig. 9 shows the marked Bunny model under various attacks.
Noise Addition: Random noise was added to all vertex coordinates (x i , y i , z i ) according to (resp.y i , z i ) where d is the average radial coordinate, and a i is a uni- We notice that even for a noise level as high as 0.25%, the correlation C(w, w ) has a value that is larger than 0.75 for all the test models expect Venus.Notice that a noise attack at the level of 0.25% will degrade significantly the visual quality of the carrier model, meaning that even though the watermark may be removed, the model will be of no use to the attacker.We also notice that the Bunny and Rabbit models appear to be exceptionally robust against the noise attack, retaining a C(w, w ) ≈ 0.92 even when the noise level reaches A = 0.25%.We believe that this is due to the fact that Dragon is the least spherical shape and thus, its radii histogram has the highest variance.
Smoothing Attack: To evaluate robustness against smoothing, we applied to the marked models 10, 30 and 50 iterations of Laplacian smoothing [42], fixing the deformation factor at λ = 0.02.The results are shown in Fig. 10 (middle).The correlation coefficients for the Bunny and Rabbit models are above 0.98 and equal to 1, respectively, even after 30 smoothing iterations.Again, the results imply that the proposed watermarking method is able to survive mesh smoothing as well.
Quantization Attack: We quantized the marked models at 8, 9, 10 and 11 bits and tried to retrieve the embedded watermark from the quantized marked models.Fig. 10 (right) shows the correlation coefficients.As expected, the robustness decreases with the level of quantization, however, the correlation coefficients are satisfactory even for a relatively coarse quantization.In particular, the correlation coefficient of the 11 bit quantization is equal to 1 in four out of the five test models.
Subdivision Attack: We subdivided the watermarked models using the open-source software MeshLab 2 .Table 3 lists the correlation coefficients for five test models.We see that the proposed watermarking is robust against the interpolating Midpoint and Butterfly subdivision schemes, with the correlation coefficient equal to 1 after three subdivision 2. http://meshlab.sourceforge.net/steps, while it is less robust against subdivision with the approximating Loop scheme.
Simplification Attack: We evaluated the robustness of the algorithm against simplification with the quadric edge collapse algorithm implemented in Meshlab.As Table 4 shows, the correlation coefficients range from 0.92 to 1, indicating that our proposed watermarking algorithm is quite robust even when 30% of the vertices have been removed.
Watermark Overwriting: Given a marked 3D model, one could attempt to embed a new watermark, overwriting the old one.From the perspective of the owner of the 3D model, the insertion of a new watermark with the same number K of histogram bins and the same origin O would replace the old watermark, erasing it completely.However, being equivalent to running the embedding algorithm twice, watermark overwriting would most probably lead to a model with higher distortion as compared to simply discarding the marked model and inserting the new watermark on the original clean model.
From an attacker's perspective, while the ability to overwrite the watermark would enable unauthorized use of the model, the main technical challenge is that both the bin number K and the origin O are unknown to them and although algorithms for estimating them can be developed, accuracy could be a problem.For example, Table .1 shows that the accuracy of the estimates of K obtained by the proposed steganalytic method is unsatisfactory.

Comparison of Watermarking Algorithms
Finally, we compare the proposed watermarking against the algorithms by Yang et al. [7] and Cho et al. [14] in terms of embedding distortion and robustness against both common malicious attacks and steganalysis.We also compared to [8] for evaluating the anti-steganalysis performance.In all our experiments, for the Yang et al.'s and Cho et al.'s methods we use the parameter settings recommended in [7], [8] and [14].
Embedding Distortion: Fig. 11 (left) shows that our algorithm generally produces lower RMSE and MSDM2 distances than Yang et al.'s [7] and Cho et al.'s [14], implying smaller amounts of distortion.
Watermark Robustness against Malicious Attacks: Fig. 12 shows that, again, our method consistently obtains higher correction coefficients than [7] and [14] when the same noise addition or smoothing attacks are applied, meaning that the watermark bits are recovered with lower error rates.
Anti-steganalysis: When testing how well the proposed watermarking method resists the developed steganalytic attack, we applied the steganalysis not only on the distribution of the means of the bins {m k : 2 ≤ k ≤ K − 1}, but also on the more relevant in this case discrete statistic of differences {|B k | − |B k+1 | : 2 ≤ k ≤ K − 1}, trying again to detect a possible bimodality on their distribution.
From Table 1 and Fig. 3, we see that the proposed steganalytic method can break Cho et al.'s original algorithms [14] with accuracy rates of up to 99%, including the cases where some bins are left in purpose without watermark bits.Regarding the watermarking method in [7], the proposed steganalysis generally does not detect watermarks with any satisfactory accuracy, the only exception being when the t-test based normality test is applied to the statistic |B k | − |B k+1 |.However, its behavior is unstable as the steganalytic accuracy varies from 80.89% for 64 bit watermarks to 96.06% for 100 bit watermarks.
Regarding the watermarking method proposed here, with accuracy rates in the region 50%-60%, the proposed steganalytic method fails to detect a watermark, no matter which of the two tests (tailor made normality test, or t-test) is used, and no matter which statistic (the means of the bins m k , or the differences of the heights of neighboring bins |B k | − |B k+1 |) is targeted.Moreover, Fig. 4 confirms that this failure of the steganalytic attack cannot be fixed by suitable choices of .
The above results show that the main objective of the development of the new watermarking algorithm has been achieved.Indeed, compared to the original Cho et al.'s algorithms, the proposed watermarking method exhibits significantly improved steganalytic properties, at least as far as resistance against the developed steganalytic attacks is concerned.
Discrete vs continuous statistics: In [14], the explicitly stated underlying assumption for the distribution of the radial coordinates is that the elements of each normalized bin follow the uniform distribution U (0, 1).Under this assumption, which is not a requirement for the working of the algorithm but it is essential for low distortion, the means of the normalized bins follow the Bates distribution To compare the suitability of the continuous and the discrete statistics to carry steganalysis-resistant watermarks we run the steganalytic algorithm on the test set of clean models with the real values substituted by random samples following the theoretical distributions of these statistics.The results are shown in Fig. 13.The x-axis indexes the models of the test set, while the y-axis shows the value of the angle θ in Eq. 6.We notice that in the case of the Bates distribution, corresponding to the continuous statistic, the values of θ have low variance and all of them are comfortably below the θ T = 2.8 • threshold we used for watermark detection.In contrast, the angles for the discrete statistic have higher variance, and reach values close to 4 • , meaning that the watermark detection test will be less accurate in classifying these models as clean.
Notice that the above test evaluates the suitability of a statistic as steganalysis-resistant watermark carrier, it does not evaluate specific watermarking algorithms.Indeed, as clean only models are used, the test could have been run at the design stage, before the development of the watermarking algorithm.However, in our case we run this test in retrospect, as a confirmation that the choice of the discrete statistic in which we based the proposed watermarking algorithm was an appropriate one.

CONCLUSION
As demonstrated by the experimental results, the developed steganalysis is able to detect the presence of watermarks by Cho et al.'s algorithms and their variants with an accuracy of up to 99%.While the proposed steganalytic algorithm was specifically designed to target Cho et al.'s two watermarking algorithms [14], the main idea could possibly be applied on several other algorithms that embed watermarks by altering a specific statistic of a histogram of the vertex set of the model.The proposed watermarking algorithm is based on a discrete statistic of the histogram of radial coordinates, the difference in the height of adjacent bins.The experimental results demonstrate that it outperforms Yang et al.'s algorithms [7] and Cho et al.'s [14] in terms of robustness against malicious watermark removal attacks and against steganalytic attacks, while at the same time it also offers some improvement in terms of embedding distortion.
In the future, we will work to develop more advanced steganalytic techniques for detecting the presence of watermark messages embedded in 3D models, and parallelly, develop 3D watermarking/steganographic algorithms that not only have better anti-steganalytic behavior, but also offer improved robustness/distortion trade-offs.

Fig. 1 .
Fig. 1.Scatter plot of the mean values M = {m k : 1 ≤ k ≤ K} (three left figures) and the variance values {v k : 1 ≤ k ≤ K} (three right figures) for the clean Bunny with K = 100 bins, and the marked Bunny with correct estimate K = 100 bins and with incorrect estimate K = 110 bins.

Fig. 2 .
Fig. 2. Q-Q plot of the mean values M = {m k : 1 ≤ k ≤ K} for the clean Rabbit (left), the marked by Cho et al.'s method with 100 bins (middle) and the marked by the proposed discrete variant of the method with 100 bits (right).

Fig. 3 .
Fig. 3. Plot of the detection accuracy with respect to the angle threshold θ T (two left figures) and with respect to the significance level α for the t-test (two right figures) for the following methods: Cho et al.'s mean-and variance-based watermarking, their variants and our proposed watermarking, where 64 (left) and 100 (right) bits are embedded into the clean models.

Fig. 8 .
Fig. 8. Close-ups of the original and watermarked Bunny and Rabbit models.For each mesh group, the left and right figures show the original and marked models, respectively.The watermarked models are those in Fig. 6.

Fig. 9 .
Fig. 9. From left to right: results of attacking the marked Bunny model by adding A = 0.50% noise, carrying out 50 iterations of Laplacian smoothing (λ = 0.02) and performing 8-bit quantization.
On the other hand, under the same assumption, in a pair of adjacent bins of equal width and with n = | Bk | + | Bk+1 | elements in total, the number of elements in the first bin follows the binomial distribution B(n, 1/2) and thus, the discrete statistic | Bk | − | Bk+1 | follows the distribution 2 • B(n, 1/2) − n.

Fig. 13 .
Fig. 13.The angles θ in Eq. 6 for the clean models with data sampled from the theoretical distributions.The angle values for the continuous case (Cho's et.al algorithm) are shown in red and the angles for the discrete case (proposed algorithm) are shown in blue.
Here, |B k+2 |/8 out of |B k+2 | elements can be moved, i.e., n right ≤ |B k+2 |/8 .Finally, we move elements into B k+1 according to the following two cases: Case 1: plots the detection rates obtained by the t-test versus the significance level α.We notice

TABLE 2
Model details and parameter setting.

TABLE 3
Correlation coefficients for various watermarked models after subdivision.We used 3 iterations for Midpoint and Butterfly subdivision and 1 iteration for the Loop's LS3 subdivision.
[14] 12. Experimental comparison between Yang et al.'s method [7], Cho et al.'s method[14]and the proposed method in terms of robustness against noise addition attacks (left) and Laplacian smoothing attacks with λ = 0.02 (right), when embedding 200 watermark bits.