ClassySeg: a machine learning approach to automatic stroke segmentation

ABSTRACT

We present ClassySeg, a technique for segmenting hand-drawn pen strokes into lines and arcs. ClassySeg employs machine learning techniques to infer the segmentation intended by the drawer. The technique begins by identifying a set of candidate segment points, consisting of all curvature maxima. Features are computed for each candidate point based on speed, curvature, and other geometric properties. These features are adapted from numerous prior segmentation approaches, effectively combining their strengths. These features are used to train a statistical classifier to identify which candidate points are true segment points. A beam search is used to approximate the optimal subset of features to use as input to the classifier. ClassySeg is more accurate than previous techniques for user-independent training conditions, and is as good as the current state-of-the-art algorithm for user-optimized conditions. More importantly, ClassySeg represents a movement away from prior heuristic-based approaches towards a more general and extensible approach.

References

1. {BSP*09} Bischel D., Stahovich T., Peterson E., Davis R., Adler A.: Combining speech and sketch to interpret unconstrained descriptions of mechanical devices. In IJCAI'09: Proceedings of the 21st international joint conference on Artificial intelligence (San Francisco, CA, USA, 2009), Morgan Kaufmann Publishers Inc., pp. 1401--1406. 6
2. {DL97} Dash M., Liu H.: Feature selection for classification. Intelligent Data Analysis 1 (1997), 131--156. 6
3. {GR00} Gotoh Y., Renals S.: Sentence boundary detection in broadcast speech transcripts. In in Proc. of ISCA Workshop: Automatic Speech Recognition: Challenges for the new Millennium ASR-2000 (2000), pp. 228--235. 6
4. {HFH*09} Hall M., Frank E., Holmes G., Pfahringer B., Reutemann P., Witten I. H.: The weka data mining software: an update. SIGKDD Explor. Newsl. 11, 1 (2009), 10--18. 6
5. {HS11} Herold J., Stahovich T. F.: Speedseg: A technique for segmenting pen strokes using pen speed. Computers & Graphics 35, 2 (2011), 250--264. 1, 2, 3, 5, 7
6. {IMKT97} Igarashi T., Matsuoka S., Kawachiya S., Tanaka H.: Interactive beautification: A technique for rapid geometric design. In UIST '97 (1997), pp. 105--114. 1
7. {Jap01} Japkowicz N.: Concept-learning in the presence of between-class and within-class imbalances. In Proceedings of the 14th Biennial Conference of the Canadian Society on Computational Studies of Intelligence: Advances in Artificial Intelligence (London, UK, 2001), AI '01, Springer-Verlag, pp. 67--77. 3
8. {JM92} Jenkins D. L., Martin R. R.: Applying constraints to enforce users' intentions in free-hand 2-D sketches. Intelligent Systems Engineering 1, 1 (1992). 1
9. {KHM98} Kubat M., Holte R., Matwin S.: Machine learning for the detection of oil spills in satellite radar images. In Machine Learning (1998), pp. 195--215. 3
10. {KK06} Kim D. H., Kim M.-J.: A curvature estimation for pen input segmentation in sketch-based modeling. Comput. Aided Des. 38 (March 2006), 238--248. 3, 5
11. {LBKS07} Lee W., Burak Kara L., Stahovich T. F.: An efficient graph-based recognizer for hand-drawn symbols. Comput. Graph. 31 (August 2007), 554--567. 1
12. {PH08} Paulson B., Hammond T.: Paleosketch: accurate primitive sketch recognition and beautification. In Proceedings of the 13th international conference on Intelligent user interfaces (New York, NY, USA, 2008), IUI '08, ACM, pp. 1--10. 1
13. {Qui93} Quinlan J. R.: C4.5: programs for machine learning. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1993. 6
14. {SSD06} Sezgin T. M., Stahovich T., Davis R.: Sketch based interfaces: early processing for sketch understanding. In SIGGRAPH '06: ACM SIGGRAPH 2006 Courses (New York, NY, USA, 2006), ACM, p. 22. 1, 2, 3, 5
15. {Sta04} Stahovich T.: Segmentation of pen strokes using pen speed. In AAAI Fall Symposium Series 2004: Making Pen-Based Interaction Intelligent and Natural (2004). 2
16. {WEH08} Wolin A., Eoff B., Hammond T.: Shortstraw: A simple and effective corner finder for polylines. In Eurographics Workshop on Sketch-Based Interfaces and Modeling (SBIM'08) (2008). 2, 3, 4, 7
17. {Wei04} Weiss G. M.: Mining with rarity: a unifying framework. SIGKDD Explor. Newsl. 6 (June 2004), 7--19. 3
18. {WPH09} Wolin A., Paulson B., Hammond T.: Sort, merge, repeat: an algorithm for effectively finding corners in hand-sketched strokes. In SBIM '09: Proceedings of the 6th Eurographics Symposium on Sketch-Based Interfaces and Modeling (New York, NY, USA, 2009), ACM, pp. 93--99. 2
19. {WSA07} Wolin A., Smith D., Alvarado C.: A pen-based tool for efficient labeling of 2d sketches. In Proceedings of the 4th Eurographics workshop on Sketch-based interfaces and modeling (New York, NY, USA, 2007), SBIM '07, ACM, pp. 67--74. 2
20. {WWL07} Wobbrock J. O., Wilson A. D., Li Y.: Gestures without libraries, toolkits or training: a $1 recognizer for user interface prototypes. In UIST '07: Proceedings of the 20th annual ACM symposium on User interface software and technology (New York, NY, USA, 2007), ACM, pp. 159--168. 4
21. {XJ10} Xiong Y., Jr. J. J. L.: A shortstraw-based algorithm for corner finding in sketch-based interfaces. Computers & Graphics 34, 5 (2010), 513--527. 2, 3, 4, 7
22. {YC03} Yu B., Cai S.: A domain-independent system for sketch recognition. In GRAPHITE '03: Proceedings of the 1st international conference on Computer graphics and interactive techniques in Australasia and South East Asia (2003), ACM, pp. 141--146. 2, 5, 7
23. {Zha99} Zhang W.: State-space search: Algorithms, complexity, extensions, and applications. Springer, 1999. 6

Index Terms

• Computing methodologies

  • Artificial intelligence

    • Computer vision

      • Computer vision problems

        • Interest point and salient region detections