Quantcast

Undocumented behaviour of "smooth" options

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Undocumented behaviour of "smooth" options

sfeam

On the bug tracker    <https://sourceforge.net/p/gnuplot/bugs/1911/>
Rainald Koch points out that when a data set contains some points with
invalid y values, the result of smoothing is unexpected. This is true
also if you use a filter to select a subset of the points, e.g.

    plot FOO using 1:( filter($2) ? $2 : 1/0) smooth acsplines

I can't find any mention of it in the documentation, but it seems that
this is intentional.  The "smooth" code has a pre-processing step that
splits the input data into N subsets separated by one or more "undefined"
points. The smoothing is then applied separately to each of the N segments
with no attempt to make the fit smooth or continuous at the junctions
between segments.

Is this documented somewhere that I haven't found?

Is there a recommended way to fit a single smooth curve to the
entire set of points after filtering?

Current CVS allows this work-around:

   set datafile missing NaN

but that is very recent.  Was there no way to handle this in the past?

        Ethan

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
gnuplot-beta mailing list
[hidden email]
Membership management via: https://lists.sourceforge.net/lists/listinfo/gnuplot-beta
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Undocumented behaviour of "smooth" options

Plotter-2
On 21/02/17 23:21, Ethan A Merritt wrote:

>
> On the bug tracker    <https://sourceforge.net/p/gnuplot/bugs/1911/>
> Rainald Koch points out that when a data set contains some points with
> invalid y values, the result of smoothing is unexpected. This is true
> also if you use a filter to select a subset of the points, e.g.
>
>     plot FOO using 1:( filter($2) ? $2 : 1/0) smooth acsplines
>
> I can't find any mention of it in the documentation, but it seems that
> this is intentional.  The "smooth" code has a pre-processing step that
> splits the input data into N subsets separated by one or more "undefined"
> points. The smoothing is then applied separately to each of the N segments
> with no attempt to make the fit smooth or continuous at the junctions
> between segments.

I can see that this will be a problem with some splines which tend to be
unstable at the end points and can veer off in either direction in a
very unsmooth fashion.

In passing, I would even question whether linking to the end points of
Bezier spline is even desirable. This seems poorly constrained and gives
unsatisfactory results. Test with a limited dataset of say 20 points to
get the idea.

This raises the question of why the code is splitting otherwise
continuous data. Is this required to reduce computation for some of the
smoothing methods, is it required for all types of smoothing algo. ?

If  this does  give unexpected results for datasets with breaks , as
reported, is this a bug or a feature?  Would a smoothing process be
expected to smooth and link across breaks in the data. Perhaps the
pre-processing should be removing NaNs non inserting them.

Should this be an option to "smooth" ?

Peter.



>
> Is this documented somewhere that I haven't found?
>
> Is there a recommended way to fit a single smooth curve to the
> entire set of points after filtering?
>
> Current CVS allows this work-around:
>
>    set datafile missing NaN
>
> but that is very recent.  Was there no way to handle this in the past?
>
> Ethan
>
> ------------------------------------------------------------------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, SlashDot.org! http://sdm.link/slashdot
> _______________________________________________
> gnuplot-beta mailing list
> [hidden email]
> Membership management via: https://lists.sourceforge.net/lists/listinfo/gnuplot-beta
>


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
gnuplot-beta mailing list
[hidden email]
Membership management via: https://lists.sourceforge.net/lists/listinfo/gnuplot-beta
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Undocumented behaviour of "smooth" options

Hans-Bernhard Bröker-2
In reply to this post by sfeam
Am 22.02.2017 um 00:21 schrieb Ethan A Merritt:

> On the bug tracker    <https://sourceforge.net/p/gnuplot/bugs/1911/>
> Rainald Koch points out that when a data set contains some points with
> invalid y values, the result of smoothing is unexpected.

Expectations can be deceiving.  :-)

> I can't find any mention of it in the documentation, but it seems that
> this is intentional.  The "smooth" code has a pre-processing step that
> splits the input data into N subsets separated by one or more "undefined"
> points.

Well, what else was the code to do with actual blank lines in the input,
or otherwise missing/unusable data?  And back in the day, invalid data
_was_ often treated as being the same thing as missing data.  Plotting a
spline across a hole in its input data would IMHO be at least as wrong
as the lack of documentation of the current behaviour.

The behaviour of the smoothing code is not that wildly different from
that of the normal 'with lines' behaviour, really: a missing entry
breaks the polyline, so it's not entirely unepected that it breaks the
smoothed curve, too.



------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
gnuplot-beta mailing list
[hidden email]
Membership management via: https://lists.sourceforge.net/lists/listinfo/gnuplot-beta
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Undocumented behaviour of "smooth" options

Allin Cottrell
On Thu, 23 Feb 2017, Hans-Bernhard Bröker wrote:

> Am 22.02.2017 um 00:21 schrieb Ethan A Merritt:
>
>> On the bug tracker    <https://sourceforge.net/p/gnuplot/bugs/1911/>
>> Rainald Koch points out that when a data set contains some points with
>> invalid y values, the result of smoothing is unexpected.
>
> Expectations can be deceiving.  :-)
>
>> I can't find any mention of it in the documentation, but it seems that
>> this is intentional.  The "smooth" code has a pre-processing step that
>> splits the input data into N subsets separated by one or more "undefined"
>> points.
>
> Well, what else was the code to do with actual blank lines in the input,
> or otherwise missing/unusable data?  And back in the day, invalid data
> _was_ often treated as being the same thing as missing data.  Plotting a
> spline across a hole in its input data would IMHO be at least as wrong
> as the lack of documentation of the current behaviour.
>
> The behaviour of the smoothing code is not that wildly different from
> that of the normal 'with lines' behaviour, really: a missing entry
> breaks the polyline, so it's not entirely unepected that it breaks the
> smoothed curve, too.
That seems reasonable. If the user has reason to believe the missing
data can be interpolated in some way, she could do that before
smoothing.

Allin Cottrell
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
gnuplot-beta mailing list
[hidden email]
Membership management via: https://lists.sourceforge.net/lists/listinfo/gnuplot-beta
Loading...