Descartes

Thus, for example, when I imagine a triangle, even though there may perhaps be no such figure anywhere in the world outside of my thought, nor ever have been, nevertheless the figure cannot help having a certain determinate nature... or essence, which is immutable and eternal, which I have not invented and which does not in any way depend on my mind.

René Descartes: Meditationes de Prima Philosophia, 1641.
[English translation: Meditations. New York: Liberal Arts Press, 1951, p.61.]

Novalis

"Die Mathematik ist ächte Wissenschaft, weil sie gemachte Kenntnisse enthält, Producte geistiger Selbstthätigkeit, weil sie methodisch genialisirt. Sie ist auch Kunst, weil sie genialisches Verfahren in Regeln gebracht hat, weil sie lehrt Genie zu seyn, weil sie die Natur durch Vernunft ersetzt." [p. 145]  

"Aechte Mathematik ist das eigentliche Element des Magiers. -- In der Musik erscheint sie förmlich als Offenbarung, als schaffender Idealismus. – Hier legitimirt sie sich als himmlische Gesandtin. Aller Genuß ist musikalisch, mithin mathematisch. – Das höchste Leben ist Mathematik." [p. 147]

"Der ächte Mathematiker ist Enthusiast per se. Ohne Enthusiasmus keine Mathematik. – Das Leben der Götter ist Mathematik. – Alle göttliche Gesandten müssen Mathematiker seyn. – Reine Mathematik ist Religion. – Zur Mathematik gelangt man nur durch eine Theophanie." [p. 147]

"Die Mathematiker sind die einzig Glücklichen. Der Mathematiker weiß alles. Er könnte es, wenn er es nicht wüßte." [p. 147/148]

"Wer ein mathematisches Buch nicht mit Andacht ergreift, und es wie Gottes-Wort liest, der versteht es nicht." [p. 148]

Novalis: Schriften , Vol. II. (Eds.: Ludwig Tieck & Friedrich Schlegel.) Fifth Edition. Berlin: G. Reimer, 1837.
Fragmente vermischten Inhalts. Part I. Philosophie und Physik (pp. 105-169).

Sylvester

One is surprised to reflect on the change which has come over the face of Algebra in the last quarter of a century. It is now possible to enlarge to an almost unlimited extent on any branch of it. These thirty lectures, embracing only a fragment of the theory of reciprocants, might be compared to an unfinished epic in thirty cantos. Does it not seem as if Algebra had attained to the character of a fine art, in which the workman has a free hand to develop his conceptions as in a musical theme or a subject for painting? Formerly it consisted almost exclusively of detached theorems, but now-a-days it has reached a point in which every properly developed algebraical composition, like a skilful landscape, is expected to suggest the notion of an infinite distance lying beyond the limits of the canvas.

James Joseph Sylvester: "Lectures on the Theory of Reciprocants" [Lectures XVII-XXIV],
American Journal of Mathematics IX, 2 (January 1887). pp. 113-161. [p. 136]

Poincaré

Le savant digne de ce nom, le géomètre surtout, éprouve en face de son œuvre la même impression que l'artiste; sa jouissance est aussi grande et de même nature.

Henri Poincaré: "Notice sur Halphen," Journal de l'École Polytechnique (Paris, 1890), 60ème cahier, p. 143.

Van Eeden

Alleen mathesis is zuiver symbolisch.

Wij kunnen ook geen wiskunstige lijnen tekenen; maar moeten het doen met strepen.

Frederik van Eeden: Redekunstige Grondslag van Verstandhouding, 1897, # 32.

Russell

Mathematics, rightly viewed, possesses not only truth, but supreme beauty — a beauty cold and austere, like that of sculpture, without appeal to any part of our weaker nature, without the gorgeous trappings of painting or music, yet sublimely pure, and capable of a stern perfection such as only the greatest art can show. The true spirit of delight, the exaltation, the sense of being more than Man, which is the touchstone of the highest excellence, is to be found in mathematics as surely as in poetry.

Bertrand Russell: "The Study of Mathematics" (1902) New Quarterly, November 1907.

Brouwer

De wiskunde is een vrije schepping, onafhankelijk van de ervaring.

L.E.J. Brouwer: Over de Grondslagen der Wiskunde. Amsterdam/Leipzig, 1907.
    

Havelock Ellis

Here, where we reach the sphere of mathematics, we are among processes which seem to some the most inhuman of all human activities and the most remote from poetry. Yet it is here that the artist has the fullest scope for his imagination. "Mathematics," says Bertrand Russell in his Mysticism and Logic, "may be denned as the subject in which we never know what we are talking about, nor whether what we are saying is true." We are in the imaginative sphere of art, and the mathematician is engaged in a work of creation which resembles music in its orderliness, and is yet reproducing on another plane the order of the universe, and so becoming as it were a music of the spheres.

Havelock Ellis: The Dance of Life, 1923, Ch. III.

Wiener

Our thesis is not that the arts are an expression of mathematics through the senses, but that mathematics itself is in the strictest sense of the word, a fine art. In this the author finds himself in complete agreement with the views expressed by Havelock Ellis in The Dance of Life. The author is fully conscious of the quicksands on which every commentator on general æsthetics must tread. However, he considers that he shall establish his point if he succeeds in maintaining the following theses: that mathematical work may produce an emotion indistinguishable from that of æsthetic contemplation; that mathematical work may and often does have as its goal the production of a work capable of exciting this emotion; that the creative mathematician is limited by the requirements of rigor only as any creative artist is limited by the nature of his medium; and finally, that mathematics has participated intrinsically in all the larger movements common to the several arts.

Norbert Wiener: "Mathematics and Art. Fundamental Identities in the Emotional Aspects of Each."
The Technology Review 32, 3 (January 1929), pp. 129-132, 160, 162. [p. 129]

Wittgenstein

Die größere "Reinheit" der nicht auf die Sinne wirkenden Gegenstände, z. B., der Zahlen.

Ludwig Wittgenstein, 1937. [Vermischte Bemerkungen. Frankfurt am Main: Suhrkamp, 1977, p. 56.]

Der Mathematiker (Pascal), der die Schönheit eines Theorems der Zahlentheorie bewundert; er bewundert gleichsam eine Naturschönheit. Es ist wunderbar, sagt er, welch herrliche Eigenschaften die Zahlen haben. Es ist, als bewunderte er die Regelmässigkeiten einer art von Krystall.

Man könnte sagen: welch herrliche Gesetze hat der Schöpfer in die Zahlen gelegt!

Ludwig Wittgenstein, 1942. [Vermischte Bemerkungen. Frankfurt am Main: Suhrkamp, 1977, pp. 83/84.]