Welcome to the story of my adventures in graduate school. I was a doctoral student in computer science at Brigham Young University in Provo Utah.
On Thursday June 4th 2015 at about 11am while running multiple experiments simulating a team of UAVs (drones) tracking a ground target, my experiments all crashed together, and I soon realized that I could no longer log in to my computer.
Shortly thereafter, I received a phone call from my advisor stating that the chair of the department wanted to speak with me. I was in deep trouble.
Normally, our department was very supportive of its students. And it was of me until this sudden change. After receiving the call from my advisor I met with the department chair. He told me that I had sent an email to a professor of mathematics asking him to read a paper I had written.
Yes, I had done so. In fact, I had asked many professors of mathematics to do that same thing. This was my crime, asking professors to listen to what I had to say and to tell me what they thought about it. One month earlier my advisor had told me that I must stop, and I had told him that I would not.
It was not easy for me to refuse to comply with my advisor's demand. I did so for deeply personal reasons.
His demand was not the first such demand. On April 15th I had received an unsettling email from someone I had not contacted who essentially demanded that I stop contacting math professors. He wrote:
"Please consider carefully the following bit of personal advice. There are two possibilities.
(1) Your proof is wrong, or (2) Your proof is right. If your proof is right, you'll be famous and successful. If your proof is wrong, you will be embarrassed and you will have difficulty getting a job. If your proof is right, you have plenty of time and the stated results are inevitable when your proof is recognized. If your proof is wrong, the fewer people know about it the better. For example, many people have embarrassed themselves by prematurely making public their wrong proofs of P != NP or P=NP. I can remember the names of some of them, perhaps you can also. Many others have also had wrong proofs of P != NP but had the good sense to show them only to one other person, or to zero other persons. Luckily for them, nobody knows how many of them there are.
It is very easy to make mistakes in mathematics. I will not try to convince you that you have made mathematical mistakes--that should be the job of your professors. You should not have made your proof public without convincing your Ph. D. advisor of its correctness. For the sake of your future career, you should now cease sending out your proof, remove it from any public website, and discuss the proof with one professor until either you see that it is wrong or you convince the professor it is right."
I wrote back later that day:
"Thank you for the very thoughtful email. I will carefully consider it carefully. Up to this point, I have chosen a different strategy.
I don't think it would be a great embarrassment to me if someone identified a flaw in my arguments. Isn't that what academics is all about, learning? Or is the purpose of academic research to build a reputation?
From the point of view of learning, the more people who consider my arguments the better. If the arguments are correct, then people will learn. If they are not correct, then people will learn why.
If my arguments are not correct, the more people who look at my paper, the higher the likelihood that someone will identify a flaw. I am willing for people to know that I make mistakes. I am also willing for people to know that I wrote the paper.
If my arguments are correct, then people will want to know sooner rather than later. If I have not made a mistake, then others have made mistakes. But isn't that OK as long as we learn from them?"
The proof referred to in the above email is A proof of the inconsistency of ZFC. On March 2nd I had submitted the paper to Philip Welch, for potential publication in the Journal of Symbolic Logic.
Conversations with several leading authorities in mathematics had not convinced me of any substantive error in my paper, but did help me better understand prevailing thought and attitudes.
I remained true to my personal convictions, at a cost. If at some future time my sacrifice proves fruitful, I will be grateful. At present, I will be patient. I take solace knowing that I was true to myself, and to my God.
I never did regain access to my computer, or to the cluster of computers in the lab of which it was a part. It was a very useful cluster. Also, I felt a connection to it since I had helped to bring in money to fund it, and had participated in its design and assembly. I missed my computer, but not only that.
What was worse than not being able to use my computer was not being able to sit at my desk in the lab where I had close contact with my advisor.
I continued to work towards completing my degree. My advisor continued to work with me, in spite of the fact that our relationship had been strained. I found a different place on campus to work, and other computing clusters.
Eventually my advisor stopped working with me. On November 1st 2016 I sent him a draft of a paper I had been working on for about a year. He read it and four days later sent me an email saying that I had done good work, which I was very excited to hear. He mentioned a few minor problems, and said that he would give me more feedback. I never heard from him about the paper again. After losing the support of my advisor, progress towards completing my degree was halted.
I later learned that I had been dropped from my program of study on November 15th (2016), because I had refused to commit to not talk with math professors.
My story would be incomplete without an account of the role that my faith in God has played, and continues to play. My faith is an integral part of my life and is especially relevant to my ZFC paper and the story behind it.
I was born in England. My father is English and my mother American, Croatian by lineage. I was raised in Fort Wayne, Indiana. I have one brother and one sister. I am the oldest and we are about seven years apart.
While I was growing up we lived in four different homes, Lawndale, Curdes, Parker Drive, and Ravinia. My favorite was the home on Parker Drive. Parker Drive was a long winding gravel road alongside a river. Our home was on about 4 acres, with a barn and two Shetland ponies. The area for the ponies was surrounded by an electric fence.
My brother Colin and I had the basement to ourselves. It was mostly one large room with a hard concrete floor. One of my favorite memories was making little bomb shelters out of legos, taking them to the top of the basement stairs and then throwing them to the basement floor. The objective was to see if we could build a structure which could withstand the impact.
We lived in a kids paradise. I doubt that it was a parents paradise. I remember an extended period of time when the well didn't work. During that time we drove to a nearby business to fill large containers with water.
During the blizzard of '78 the city shut down for three days. School shut down for a week. I decided to use that time to hone my fire-making skills.
My church sponsored a Boy Scout troop. We went on a 50-mile afoot afloat adventure. We canoed for days, drinking in the beauty of the river and engaging the other canoes in battle. At night I was eaten alive by mosquitoes, but that discomfort didn't begin to dampen my enthusiasm.
One family vacation we stayed for a week in an old cabin in the backwoods of Kentucky. Before we could use the cabin, spider webs and other traces of animals had to be cleaned out. Mom brought beautiful quilts from home for the beds and cooked on an old wood stove in the center of the cabin.
I played endless board games with Dad. The aim of my existence that week was to beat him, but I never did. At one point I broke down and cried after losing a game I had so hoped to win. He didn't like seeing me hurt and asked if he should let me win. I choked down my tears and said something like "No Dad, it's OK. I don't want you to let me win."
I delivered newspapers to a nearby apartment complex. I enjoyed delivering the papers. It was good exercise and I felt good about learning to work. I didn't like collecting money from my customers. I collected enough to pay my bill to the newspaper company and a little extra for myself, but there were many people who got a free newspaper.
I attended early morning seminary which helped me to learn and appreciate the scriptures. I felt the love and faith of my teachers, and I grew to love them.
I played the trumpet in jazz and marching bands. We traveled to Daytona Beach in Florida. The rising sun on the ocean was spectacular.
I remember my teachers Mr. Beights, Mrs. Szink, Mr. Wittenberg, Mr. Dvorak, Mr. Ashton and Mrs. Thomson. They instilled within me a love for learning and a confidence in my ability to succeed in life.
I did enjoy high school, but when I finished I felt free. Free to pursue a life of challenge and achievement.
Both of my parents had gone to BYU. That's how they met. It made perfect sense. I'd go to BYU too, find a beautiful girl, and marry in the temple for time and all eternity. I'd be a dad, gain experience, and learn to become like my Heavenly Father. So off I went, across the country to a parched land with no trees, no mosquitoes, and beckoning mountains.
It was difficult to be a responsible adult. I was just starting out, and learning important lessons through hard experience. Surprisingly, my grades were only average. More was being expected of me, and I began to realize that I had wasted opportunity in high school due to my poor attitude toward studying.
I found some awesome friends. We had a great time together.
I received my patriarcal blessing from Clyde Sandgren, which inspired me with a powerful vision of what I could become by following the gospel plan.
My boyish heart was broken. My friends patiently listened. Time helped cheer me up.
At the end of the academic year, it was time to return home to Indiana. The first order of business was to earn money for a mission. I got hired by a landscaping company, but couldn't actually work for the first two weeks due to heavy rain. But I was determined to work. So I decided to start a television repair business, to pursue when I wasn't landscaping.
Mostly I just replaced worn out vacuum tubes, but there were other problems. One in particular stumped me. The main fuse was blown. After replacing it, the new fuse blew immediately. I had no training in electronics, and no idea what I was doing, so I decided to wrap the blown fuse in tin foil. KA-POW! That was exciting, and distressing. Obviously there had been something else wrong, causing the fuse to blow. Unfortunately, the TV was now dead as a doornail.
After a year and a half I didn't have much money for a mission. I had some expensive TV repair equipment, which I planned to use afterwards. My father realized what was happening and decided to charge me a little each month for rent. Without saying anything, he saved that money for me. When it was time to serve the Lord as a missionary, my dad let me know what he had done. I would have enough.
I received a call to serve in the Germany Hamburg mission, of the Church of Jesus Christ of Latter-day Saints. As part of my preparation I returned to Utah and received my endowment in the Salt Lake temple.
My mission transformed my life. That miracle unfolded through service, service to the Lord and to my neighbor. It was a difficult form of service, difficult to give because it wasn't welcomed. I became a new person because of it, less selfish and more dedicated to the Lord.
Rather than share details of my mission in Germany, I'd like to share a scripture. "And if it so be that you should labor all your days in crying repentance unto this people, and bring, save it be one soul unto me, how great shall be your joy with him in the kingdom of my Father!" (Doctrine and Covenants 18:15)
Following my mission I returned to BYU. I was a new person, but still far from perfect. I would describe this period of my life as a bit painful. I am grateful for the influence of friends from my mission who continued that friendship afterwards. One of those friends and I ran the Saint George marathon together. That was fun.
I was blessed to marry a bright beautiful girl, my best friend. Together, we were blessed with Devon, Joshua, Lauresa, Brian, Christopher, Adam, Amelia, Samuel, Alison, John, and Michelle. And the blessings continue to come as our children marry and have children of their own. Truly the windows of heaven have been opened and blessings poured out upon us.
While in school, I taught German at the missionary training center and took care of equipment in the electrical engineering shop.
Around the time I graduated in computer and electrical engineering, I decided to venture off on my own. I wanted independence and adventure. There was so much I looked forward to doing and learning. I was brimming with enthusiasm and self-confidence.
One of my ambitions was to use computers to help people learn foreign languages, particularly to help with listening comprehension and pronunciation.
Most computers at the time lacked sound capabilities. For my senior project I designed and built a sound card together with amplified speaker and enclosure for the IBM PC.
My plan was to start by repairing electronics, and then use my spare time to branch out into other areas, such as developing educational software and whatever else I wanted to do. So I set up shop in our apartment and went to work.
My business specialized in computer monitor repair. When a monitor stopped working, I would find the diode, capacitor, or other component that had failed and replace it. Often the problem was a bad solder joint. The cost in materials to fix a monitor was generally very low. The challenge was to quickly find the faulty component, or components.
I learned that time was better spent when I actually understood how a circuit worked. I rarely had a schematic diagram to help. Rather, I studied the layout of the circuit board. This practice of learning how a circuit worked by studying the layout of the board was fairly practical since circuit boards were almost always single-sided.
If I didn't find the root cause of a problem at first, I would need to go back and take a closer look until I found the culprit perpetuating the problem.
My family was growing, my business was marginal, spare time was scarce, and money was tight. I began to realize my limitations. Instead of being buoyed up with anticipation for the future I began to be weighed down with failings of the past.
Obviously, things weren't going as planned. At that time I had two small repair shops, one in Provo and the other in Auburn Washington. I thought it would be best to leave my business and join the growing software development market back in Utah.
Before making a final decision I counseled with the Lord through fasting and prayer.
Earlier, before starting my business I had done the same. At that time I really wanted to start my own business and essentially asked the Lord for permission. The answer I received was that it would be fine with Him, as long I as lived righteously and kept His commandments. So, I enthusiastically started my business.
This time was very different. I asked for counsel regarding my plans to change course. The answer I received was that it would be better for me to stay with my business and discipline myself to run it effectively. I was counseled to pay closer attention to detail and to start work consistently by 8am.
So I stayed with my business and put my effort into doing a better job.
I tried working more. I put more burden on my sweetheart at home. I was negligent fulfilling my responsibilities at church. As scoutmaster, I chose not to go with my troop to summer camp. Though, I did not work on Sunday. Finally, I realized that working too little was not the problem.
I tried working smarter, and saw limited success. Gradually, I was doing a better job.
And I was growing in other ways. I was asked to serve as scoutmaster again, this time back in Utah. Smarting from earlier poor choices, I decided to serve wholeheartedly and felt great about it. My troop helped build Discovery Park in Pleasant Grove.
But in spite of improvements my business was a dead end. I could not find a way to get a decent return on the investment of time required to diagnose a failure. There were too many different failures. There were too few monitors with the same failure. And, the cost of a new monitor was steadily falling.
I had tried many times to start work consistently by 8am, but had not actually done it. About seven years had passed, since I received that counsel from the Lord. I knew that starting work early was important, but did not think it was going to solve all my problems.
One Sunday while sitting in church partaking of the sacrament I promised the Lord I would do what He had counseled me to do. I realized that I had been trying harder to do my best than to follow His counsel. During the next few years I wrestled to keep that promise, and eventually did. Keeping my promise to God meant everything to me.
I learned that it was more important for me to listen to my Heavenly Father than to succeed, more important even than to succeed in doing something good. I realized that by choosing to follow His counsel I had embarked on a spiritual journey to a land which could only be reached with His continued help and direction.
From the moment I made that promise I was filled with a profound sense of peace and happiness. The Spirit of the Lord whispered to my soul that God was pleased.
While fixing monitors I didn't have any free time on my hands, not free time for my hands. My hands were always busy. There were things to take apart, to put back together, to solder, and to un-solder.
But my mind had free time. The challenge was how to take advantage of that opportunity. I wanted to learn Java. I figured that I needed to read the language specification. There were other things I also needed to read, such as the scriptures and teachings of the presidents of the church.
So I wrote a program to read these things to me aloud, while I worked.
My mind had free time, but that time was frequently interrupted. The phone would ring. A customer would come in. I would finish what I was doing. Another challenge was understanding the output of text-to-speech software. Some words were pronounced incorrectly. The output was unnatural, lacking the natural flow of speech which helps the listener understand.
So I wrote the program to read aloud only one line of text at a time, then wait for input from me. That way, I could take a phone call, help a customer, or think about what needed to be done without having to pause the computer. Also, if I had trouble understanding what was said, I could easily have the computer say it again.
I came to really like the fact that the computer waited for input before continuing. Perhaps what I liked most was that I could easily think about what I had just heard. My pondering was not interrupted by the computer. Reading continued only when I was ready. Reading by listening in this way was more like regular reading, which only continued when I took initiative by moving my eyes to the next line of text.
I had the idea that the program I had written for reading could be modified to act as an audio computer interface, enabling a person to use a computer without a visual interface. I thought an audio interface might be useful to people who were driving or otherwise unable to look at a visual interface. I loved to hike and think. Perhaps I would enjoy having computer access without the necessity of stopping to look at a screen, or trying to view a screen while walking.
I realized that a simple touch-tone telephone had the necessary electronics to implement a basic audio interface: a network connection, a speaker, and some buttons. And it had a microphone.
I decided to pursue my audio interface idea by building a prototype. I figured that writing the necessary software would either help me start a new business or help me start a new career.
So I built a prototype using a Dialogic telephone interface computer expansion board. The basic functionality was that a user could peruse a set of choices by going forward and backward through a list of audio messages, and then initiate an action by selecting the desired choice. The telephone keypad acted essentially as a computer mouse, and the telephone speaker acted as the display.
A friend helped me find a market for the invention, which I pursued. A local movie theater decided to use the system, dubbed Keypad Browser, to announce movie showtimes over the phone. The system allowed users to quickly find showtimes they wanted to hear rather than having to wade through one long message. My dad and brother-in-law encouraged me to file a patent. I wrote it, and my dad paid the filing fee.
At this point my monitor repair business had dried up. I took some odd jobs, including delivering newspapers with the help of my oldest children. My brother offered me a job at a computer hardware startup which was being acquired by Avocent, and I took it gratefully. Cinemark, a movie theater chain, expressed interest in my product and asked me to send them one for a new theater opening in American Fork.
During the next two years I got a taste of working for a large business, which I thoroughly enjoyed after being on my own for so many years. It was great to work together as a team to try to bring a product to market. My ability to diagnose difficult problems came in handy several times.
During that same period, Cinemark bought many of my movie showtime announcement systems, and I successfully defended my patent claims with the help of a law firm.
Suzanne and I focused on paying off debts. We felt extremely blessed. We had just finished paying off everything except the mortgage when news came that my employer was shutting down its branch in Utah.
Our house was small and brimming with children. We had a large back yard for kids to play and Mom to garden. We grew tomatoes, peppers, green beans, corn, onions, raspberries, apples, peaches, and a few potatoes. We tried to grow apricots and pears, but without much success. Each fall we canned enough salsa to last a year, around 60 quarts. We also made our own bread, as much as 16 loaves every two weeks from freshly ground whole wheat.
There were plenty of opportunities to develop Christ-like attributes, such as patience and brotherly kindness, especially with seven brothers in the house! We shared one bathroom.
In the winter I split logs for the wood stove. It was big and put out a lot of heat. The house was poorly insulated. The wood stove helped keep down the heating bill and made at least one room very warm. The light from the fire cheered Mom's spirits when winter gloom blocked out the sun for extended periods.
Mom instilled a love for reading and music in each child. She regularly took them to the library and read to them. She kept an eye open for a good used book at the local thrift store. We limited time for media in our home so that there would be plenty of time for homework, reading, singing, and visiting together. She taught each of our children to play the piano. The sound of music filled our home.
Mom cut everyone's hair, except her own.
After a long day of hard work, Mom would settle down and enjoy reading for herself. We loved her, and she loved us. We tried to be good and helpful so she would not follow through on her repeated threat to run away to Mexico.
My wife helped me to see that I needed to change the way I treated my oldest son. The Spirit of the Lord softened my heart and helped me humble myself sufficient to listen and understand the truth of what she said.
Things were going great! Then in November 2005, my company decided to close its newly acquired Utah location. With that closure came a career decision for me. I enjoyed my work there, but was not excited about relocating to Alabama or pursuing a similar type of work with some other company. I felt I needed to pay attention to my career, not just to getting another job.
I was inspired by the thought of people benefiting from the system for reading by listening which I had discovered. My Keypad Browser audio computer interface technology could help provide such a benefit. Up to this point in time, the technology was merely being used to provide efficient telephone access to movie showtimes. Its full potential had not yet been realized. I thought perhaps it was possible to build a business on Keypad Browser technology. I attempted to put together a team and a business plan. I studied marketing and participated in the Utah Valley Entrepreneurial Forum.
In August 2006, I received a church calling which included a speaking assignment in front of over a hundred people, almost every month. Oddly, I wasn't afraid to speak in front of so many people. However at first, I was almost incapable of preparing a coherent talk. I found it extremely difficult to decide what to say, and how to say it. I stayed up all night trying to prepare for my first speaking assignment, making almost no progress. I was fried going to the meeting, and totally unprepared. Lucky for the congregation, most of them were residents of a care center who had difficulty hearing or understanding what I said.
By January 2007, with the help of friends, I learned that it was not practical for me to build a career by building a business. I needed a different career plan. While out for a run, I had a strong spiritual impression that it was time for me to run as fast as I could, and that doing so involved serious risk.
I contacted TellMe, the company which had won what used to be my Cinemark business and told them about Keypad Browser technology, and how it might help them. Perhaps they would be interested in licensing my patent, or working with me in some way.
I flew to California to pitch my ideas. That trip did not appear successful. Yet, it was another spiritual turning point in my life. I felt that God was especially close. I felt that He was interested in me, and in my concerns.
On the morning of March 1st 2007, during my prayer to begin the day I received the clear instruction: Go to BYU. I acted in faith and came to understand that the Lord wanted me to set my sights on a PhD, and consider the possibility of being a university professor.
I found a friend in Deryle Lonsdale, a professor at BYU who was actively involved in computational linguistics research. He invited me to participate in his PSST (Pedagogical Software and Speech Technology) research group. I began attending meetings and trying to learn.
I thought I might use what I learned to improve my reading by listening system. Perhaps I could figure out a way to automatically detect paragraph boundaries in speech, to break up speech into small segments, or ideas, which then could be listened to thoughtfully, one at a time.
I searched the Harold B. Lee library and found a periodical named Computational Linguistics. I started skimming articles to try to get a feel for the field. Mostly, I understood very little. But, I did start to get interested in some questions.
Dr. Lonsdale introduced me to a professor in the computer science department who also was interested in computational linguistics, Eric Ringger. Thanks to both of their kindness, I continued to learn and progress.
Dr. Ringger invited me to sit in on his statistical natural language processing course. I was totally fascinated! After the end of the course, I asked if I could sit in on another. He let me know that I needed to enroll as a student.
So, I applied to the Master's degree program in computer science with Dr. Ringer as my advisor, and miraculously was accepted. I was SO excited. I was embarking on a new career, belatedly but finally where I belonged.
Upon entering the program in the Fall of 2008, I took a part-time job grading papers for another professor in the Computer Science department, Kevin Seppi. I thought it was pathetic that I was working for so little money, but I was grateful for the employment and for the opportunity to learn more about the work of education.
My first experience trying to publish was with Dr. Lonsdale and another member of the PSST research group, Jeremiah McGhee. We pursued the idea of predicting scores in oral language proficiency interviews, using scores from a fully automated sentence repetition test. People learning English were tested in both ways, in an oral interview, and using an automated system. The automated system played a recording of an English sentence, recorded the test-takers attempt to repeat the sentence, and then scored the attempt. The automated test was not nearly as comprehensive a test as the interview, but it was much less expensive. We found that we were able to use the automated test to predict interview scores, to some degree.
Our first attempt to publish was a last minute affair, staying up late into the night to write a first and final draft. Our results were promising, but I was not at all confident in my ability to write. That first submission was rejected. Among other reasons, we had not cited an important related work.
Over the following year and a half, two revisions of that first paper were submitted and subsequently rejected. Dr. Lonsdale had kind of given up on the paper. But in April 2011, two years after our original attempt, we succeeded! This success was a big confidence booster, for me. The experience helped me to learn about the whole process of researching a question of interest to others in a global conversation of ideas, designing and running experiments, analyzing results, and writing to contribute to the conversation. I learned a lot about writing. I learned to express myself so that others could understand what I wanted to communicate.
While working on my oral proficiency paper, I also worked on my thesis. I had participated in research groups with both Dr. Lonsdale and Dr. Ringger. Dr. Ringger's group was studying spoken language identification, automatically determining which of several common languages of the world is being spoken in recorded speech. I decided to address the language identification problem for my Master's thesis. I became interested in understanding what was required to obtain an arbitrarily high correct classification rate.
Initially, I worked with spoken language. One popular method for identifying language was strongly influenced by commonly spoken words. I wanted to be able to recognize a language even when the words spoken were not common.
On July 13th 2009 I recorded: "I went to the temple Saturday and received some guidance regarding a model to generate language. My language model can contain my training data, all in order. I can generate language by reproducing the training data, all in order. This would represent the highest level of correlation with the data. Lower levels of correlation could be obtained by introducing discontinuities. If a generated sequence only has one discontinuity, that means that it can be divided once, to form two sequences, both of which are sub-sequences of the data. A measure of similarity to the language would be the approximate number of discontinuities in a sequence divided by the length of the sequence."
That idea came to me in the temple. The Lord helped me in my research. He has helped me all along throughout my life, though much of His help has come by way of letting me struggle, without providing the specific help I thought I needed.
The idea consisted of generating language by concatenating segments of the training data. The average length of the segments could be used as a similarity measure, similarity of the generated language to the language of the training data.
To illustrate, training data might be a newspaper. A segment might be a sequence of characters from the newspaper, such as part of a word, a whole word, part of a sentence, and so forth. Suppose that the newspaper contains "the dog barked so loudly that". Now suppose that I want to determine if the sequence of characters "the dogs barked" belongs to the same language as the newspaper. I could generate that sequence by concatenating segments found in the newspaper: "the dog", "s", and " barked". The average length of these segments is 5 characters, which could be used to measure similarity between "the dogs barked" and the newspaper.
Prevailing measures of similarity gave high scores to sequences containing segments which occur often in the training data, while this measure ignored the popularity of segments and only considered their length. I hoped to show that this measure would prove useful for estimating the probability that a sequence belonged to a language.
The basic innovation I proposed was not specific to spoken language. I figured it could be applied to either spoken or written language, text. If the innovation had value, I should be able to demonstrate that value using text. It would be much easier to run experiments with text, rather than with audio. Also, the field of text classification was more general and widely studied than the field of spoken language identification. So, I decided to first try to validate my ideas using standard text classification tasks.
By the end of the year, 2009, I had some positive experimental results, something Dr. Ringger kept reminding me I needed to have!
Dr. Ringger worked with Hal Daume from the University of Utah to organize an NLP (Natural Language Processing) Fest for their students, to be held at the end of February (2010). In preparation for this fest, he invited his students to present their research to each other. I struggled to prepare my practice presentation. On Wednesday, February 3rd, I wrote: "Yesterday I presented a preliminary version of my thesis proposal to the NLP lab. It went very well. I was working on it on Monday, but was struggling a lot and not making much progress. It is important for me to remember to call upon the Lord in all of my concerns. I did that, and the next day I was able to make much better progress. I felt that it would be a good idea to fast because I was presenting that day. So I did. While fasting, I was reminded that I could fast when I needed help with something especially difficult for me. I will remember to do that in the future."
In retrospect, I would like to share a scripture from the Book of Mormon. I don't remember if this specific scripture came to mind when I had that experience, but it most likely did. The verse is Alma 34:20 "Cry unto him when ye are in your fields, yea, over all your flocks." I had come to understand that is was OK to ask God for His help with things like my work, things which one might not think would rise to the level of Divine concern. Yet at the same time I found myself forgetting the principle, repeatedly trying to do too much on my own. I was trying to do a lot. I was married. We had eleven children. Our youngest was not yet three years old. Suzanne and I both actively served in our church. We tried to be good neighbors.
I did receive much help from Heaven, particularly that month. I had several key insights into my research. I realized that the fewest possible number of segments from training data which could be concatenated to generate a sequence was generally the most likely number of segments to have randomly been drawn (to generate the sequence according to my model), and that it was possible to determine that number very efficiently. Dr. Ringger told me about a certain type of data structure called a suffix trie, which could help to calculate the number.
By this point in time, I had essentially completed the required course work for my Master's degree. I got so much out of those courses! I especially enjoyed the course on Bayesian methods I took from Dr. Seppi.
And by this same time I was finally making good progress towards writing the required thesis. If my goal was to earn a Master's degree, I should have been close to finishing. But my goal was to earn a doctoral degree. That is what I had felt so strongly that I should try to do, before going back to school.
In my letter of intent, as part of applying to the Master's program, I wrote that I intended to continue on to complete a doctoral degree.
In May of 2009 I had asked Dr. Ringger if he would be my PhD advisor. I understood that I would need someone willing to be my advisor in order to be accepted into the doctoral program. He had told me that doing so would be a big commitment for him. His initial answer was no, but it might change if I did good work later on my thesis.
On January 26, 2010, I recorded in my journal an impression I had received, that I should plan to switch advisors from Dr. Ringger to Dr. Seppi.
On March 9th I talked again with Dr. Ringger about being my PhD advisor. I thought I had made good progress on my thesis and that he might be willing to change his mind. He was not willing at that time.
Remembering that I had received an impression that Dr. Seppi would be a great PhD advisor for me, I asked him the following day. He didn't say yes, or no. I had no assurance that he would later agree to my request, except for the spiritual impression I had received.
On April 9th I recorded: "I have decided to ask Dr. Seppi to be my adviser now, rather than later. Dr. Ringger has been very helpful to me in the progress which I have made to this point. It is now time for me to begin working with my future PhD adviser."
A few days later I told Dr. Ringger about asking Dr. Seppi to be my advisor.
Dr. Lonsdale, Dr. Ringger, and Dr. Seppi were not the only ones who greatly influenced my experience in graduate school. So also did my fellow students. One experience is particularly memorable. I didn't record anything about it at the time, so I don't know when it happened. My guess is that it occurred in the Fall of 2009.
One day in the NLP lab I made an off-hand joke about evolution. (I used to do that sort of thing because the idea of our beautiful world being created by chance struck me as funny.) I do not remember what I said, but the response surprised me. The response was that my lab partners did not think my joke was very funny. One of them said that the evidence that man evolved was overwhelming. I do not think I responded. I had no idea what to say.
This experience had a great impact upon me. I felt that something was terribly wrong. I felt a need to do something to help. I wondered why God had led me back to BYU. Originally, I thought that He was helping me to establish myself in a career that would be right for me. Now, I wondered if there was some special service He wanted me to perform. My patriarchal blessing, received while attending BYU as a young man, gave me knowledge about future responsibilities the Lord would have me fulfill. I thought about that blessing and how it related to this experience.
The idea came to me that I could use my model for generating language to randomly generate child DNA sequences from parent DNA sequences, and show that it is more likely that human DNA is a parent, rather than a child sequence when compared to other DNA.
Armed with a new sense of purpose, I moved ahead with my studies. I planned to include a computational biology course in my future curriculum. I looked for ways to link my study of language to the study of DNA sequences.
One link was probability. I already was learning about the probability of sequences of characters in text. Since DNA sequences are also represented as sequences of characters, I could apply what I learned about probability of sequences of characters in language to help me learn about probability of sequences of characters in DNA.
During the Summer of 2010 my advisor left to join his wife leading a study abroad tour for BYU students. Prior to leaving he was quite busy helping his other graduate students complete important tasks prior to his departure. I was pretty much on my own for several months.
When he returned I asked him if I could change my research area from text classification to language modeling. Text classification is the problem of classifying, assigning a class, or label, to a sequence of words. Language modeling is the problem of assigning a probability to a sequence of words, which probability can be used to classify the sequence. He did not think it was a good idea to make such fundamental change so late in the course of my studies, but he let me do it.
I was relieved that he let me pursue my own path, even more independently than before, but I didn't anticipate how difficult it would be for us to continue to work together. I didn't fully realize that bad feelings were damaging our relationship. Yes, I had told him that I wanted someone else to be my advisor, and I sincerely did, yet I didn't consider the possibility that I might irritate or offend him.
The fact that I wanted a different advisor was not the only irritant. I also wanted to play the role of teacher, instead of student. I wanted to explain to him what he wasn't understanding.
I appreciated his feedback, suggestions, and critique. They were invaluable to me and I couldn't have progressed without them. Yet, his most critical objections also revealed a lack of understanding. His objections helped me to better explain my work, I thought.
My desire for him to understand me eclipsed my attention to his feelings. He simply was not comfortable with me or my research. He felt it was inappropriate for me to be teaching him. He didn't know exactly where I was mistaken, but I had to be. I was claiming that not only he but also an entire academic community had somehow missed something important, which I could help them see. That was extremely unlikely, and he was not excited about untangling whatever confusion mislead me. In my naive exuberance I would surely embarrass us both, he felt.
In the Fall of 2010 I started taking classes for my PhD program, even though I was not yet admitted to that program. All of my Master's program classes were finished, but my Master's thesis was not. My Master's thesis proposal still lacked approval.
Dr. Seppi started getting involved in my research. Both he and Dr. Ringger asked challenging questions.
On April 26, 2011 the oral proficiency paper I wrote with Jerry McGhee and Dr. Lonsdale was accepted for presentation at the 6th Workshop on Innovative Use of Natural Language Processing for Building Educational Applications. Yay!!
Just 50 days later Dr. Ringger approved my thesis proposal. Success in my paper with Jerry and Dr. Lonsdale meant it was more likely that I would also succeed in my Master's thesis.
Also during Winter semester, I took a course in Computational Biology taught by Mark Clement in the computer science department, with some help from the head of the biology department.
My journal entry of July 1, 2011 reveals some insights into my thoughts, feelings, and experience at that time:
"There is so much I haven't written. I did recruit Dr. Lonsdale and Jerry to help, and the OP submission was very successful. I just returned from ACL-HLT BEA (Association for Computational Linguistics - Human Language Technologies conference, Building Educational Applications) workshop were I presented our paper. Between Dr. Lonsdale, Ringger, and Seppi, all travel expenses were paid. I decided to pay for food. This paper represents a pivotal accomplishment for me in my new career. The Lord has blessed me greatly.
Also, my master's thesis proposal is officially complete. On the morning of March 15th, I asked Suzanne to fast with me. I had been trying to write a needed revision of my thesis proposal, without much progress. At the same time I had heavy deadlines looming, with no room for any delays. My standing in the department was in jeopardy, my two winter semester classes were being neglected, and I had yet to revise and submit the oral proficiency (OP) paper. After our fast I was able to write. I wrote and wrote, and wrote well. I was so blessed. The Lord is helping me to understand that he wants me to turn to Him for help, rather than try to do everything own my own. I am learning that He will help me, that I must rely on Him.
I wrote a research paper in my computational biology class on a subject which I am planning to pursue as a PhD thesis, which is: In the segment selection model, the DNA of man is a more probable universal common ancestor than the DNA of a simpler life form. I have spoken with Dr. Clement about this and he believes that no scientist in the world will believe me, unless perhaps I am able to build sufficient credibility. He gave me some ideas as to how I might do that. He also advised my to not discuss my thesis with others, at least until some credibility in this area is established. I wonder if that request extends to my advisor."
Now that the thesis proposal was approved, it was time to have fun writing code and running experiments! However, it didn't take too long before I started to flounder. It was not as easy to prove my thesis as I thought it would be. Instead of enjoying smooth sailing, I worked up a sweat bailing. I got some promising results, but I also got results which seemed to go against what I was trying to prove. There was a lot more to learn than I expected, and it took much longer than I planned. I ran gazillions of experiments. Truly the space of possible experiments is unbounded, infinite.
I did gradually overcome obstacles and find ways to show what I wanted to show. By March of 2012 I was feeling fairly good about experimental results.
Below are a few more selections from my journal:
March 5, 2012
"Last Thursday I met with Dr. Ringger and explained why, in speech rec and mt, we use language models as if they estimated a prob of belonging. He expressed incredulity that I could uncover a fundamental error in the field of language modeling and challenged my judgement for believing that I could do so. I was surprised that he gave such direct voice to these concerns. I have long suspected that he had those concerns, but this was the first time he directly expressed them. I am happy that he has done so, because it will help improve our communication.
Both my professors, Dr. Ringger and Dr. Seppi, have continued to express disbelief of the primary claims I make to motivate the new model I introduce. I have already given many reasons to explain my motivation, but I realized last week that the reasons have not been sufficient for my professors. On Saturday I realized that I could prove my motivational claims, and did so in a two sentence email. I am beginning to understand that I need to prove all my claims. Now that my motivational claims have been formally proven, I believe that my professors will soon approve my work."
March 14, 2012
"Dr. Ringger said he is coming around to agreeing with me on a main point I have been making in my thesis! I am diving into the study of DNA sequence evolution, as part of my planned doctoral work. I am doing that first, then refining my master's thesis, each day."
April 19, 2012
"At the beginning of this month I had two key meetings, one short one with Dr. Ringger and Dr. Seppi, and one long one with Dr. Seppi. For both of those meetings I prepared ahead of time to directly address their specific concerns. To prepare for the meeting with Dr. Seppi, I spent an entire day pondering how best to do that, and during the meeting it was evident that I had succeeded, what an important blessing.
Today, I had a good talk with Dr. Seppi. He outlined three important objectives for me to accomplish this summer, repair my relationship with Dr. Ringger, help a particular funded research project to succeed, and submit a paper based on my master's thesis. He said that if I can accomplish these things, I will probably be able to be accepted into the PhD program in the fall. If I don't accomplish those things, I probably won't be accepted. I am planning to accomplish them and am very happy to have this opportunity to succeed.
Starting very soon, I am planning to work for Dr. Seppi on a funded research project. The project is particularly suited to my expertise in language modeling. This work will give me significant opportunity to advance my new academic career, but it will also postpone the biology-related thesis I am planning to pursue, which is OK because it will better prepare me for that future work.
I am beginning to see and believe that my job is to do the best with the opportunities that I have, turning down those which will seriously distract from the direction I need to go, and pursuing those which will help, realizing that important opportunities may be very unexpected. I will expect that unexpected opportunities will enable me to fulfill my responsibilities in mortality."
May 14, 2012
"This morning Sue fasted together with me, to help me in my meeting with Dr. Ringger. The meeting went very well. I believe that the email I sent on April 27th, just after our last meeting, did indeed address his primary concern with me. Now, the problem between us has been solved. This is such a great blessing. The scripture comes to mind, "Howbeit this kind goeth not out but by prayer and fasting." (Matthew 17:21)."
May 16, 2012
"Yesterday was the magicc kickoff meeting, related to my new work which I am starting full-time as of today."
July 18, 2012
"Brian died in early morning"
Brian is my son. He passed away peacefully in his sleep, for no apparent reason. He was strong, emotionally and physically. He had recently received a call to serve as a missionary in Mexico. I love and miss him. I am grateful to God for the privilege of being his father. I will see him again. I have no doubt of that!
Shortly after Brian died I found a much simpler way to prove my thesis. The experimental results were clear. I decided to stick with those results. I knew that the Lord had helped me.
I learned something from those results which amazed me. In my thesis I argued that if a probability that a sequence belongs to a language is desired, then it is important to avoid assuming a degree of generalization. I proposed two methods for avoiding such an assumption: MINS and Segment Selection. MINS was very simple and fast. Segment Selection was sophisticated and slower. I assumed that Segment Selection would be more effective, but it wasn't. MINS was better! I figured that this fact was significant and planned to base my doctoral research on it.
MINS (Minimal Number of Segments) is the minimal number of segments required to generate a sequence of characters Y by concatenating segments found in a sequence of characters X, denoted as mins(X→Y), where each segment is a contiguous sequence of characters.
The kickoff meeting was the start of a research project funded by the US Air Force to learn more about using UAVs (Unmanned Aerial or Air Vehicles) to intelligently and cooperatively track ground targets in an urban environment. BYU had been selected to help with this research project because of the MAGICC lab led by Randy Beard from the electrical engineering department and Tim McLain from the mechanical engineering department. Dr. Seppi (from the computer science department) got involved, and then he invited me to share my ideas about how research in language modeling could be applied to this urban target tracking problem. The urban environment meant that there would be tall buildings which would sometimes prevent UAVs from viewing the ground target. A similar situation occurs in language modeling when an unknown word is part of a sentence. An unknown word is similar to a period of time in which the ground target is not observed by UAVs overhead.
Most of my time was spent working on the UAV research project with Dr. Beard, Dr. Seppi, and others. I spent a little time working with my advisor to finish up my thesis so that I could earn my Master's degree. All of the courses required for both the Master's and Doctoral degrees had been completed.
The UAV research project was in Phase 1, with funding for six months. All parties involved hoped to extend that research into Phase 2, which would include much more significant funding and last two years. I played a leadership role, which was super fun, although some of it surely went to my head.
On November 7 my advisor decided to postpone plans for my thesis defense. He thought I had overlooked a critically relevant field of study. I told him that I would look into it. I did, and argued that the field was not as relevant as it appeared.
I also spent time preparing for my doctoral research by investigating the usefulness of MINS in the study of DNA. I noticed that a certain paper published in 1992 by David M. Hillis and others had become influential by creating in a lab a known phylogeny of viruses. The researchers took a virus and then mutated it to create two slightly different versions of the original virus, and then repeated that same process for a few generations. The researchers then applied models for predicting evolutionary relationships given data about the DNA of the viruses in the known evolutionary tree.
The models used to predict evolutionary relationships ignored certain information which I knew that MINS did not ignore. I wondered if MINS could do a better job of predicting than the other methods. I found that MINS was useful for some things and learned more about how to use it.
Phase 1 had ended which meant that I was no longer getting paid. But, our research group was still quite active preparing a publication on our Phase 1 results and preparing to apply for Phase 2 funding. I suggested ways to use MINS in our Phase 2 proposal.
On February 5 (2013) I met briefly with my advisor and gave him a paper copy of a complete draft of my thesis.
A few days later, I wrote a note to myself about using MINS to infer information about an unknown common evolutionary ancestor of two observed DNA sequences of the same length. Assuming that the two observed sequences evolved through random mutation from a common ancestor it may be possible to infer whether the common ancestor is more closely related to one of the two observed sequences. I had performed experiments using data in the 1992 Hillis paper demonstrating effectiveness of the following method, which I describe here as:
Calculate MINS from observed sequence B to observed sequence C, mins(B→C). Then do the reverse, calculate MINS from observed sequence C to observed sequence B, mins(C→B). If those two MINS values are similar and low, then the common ancestor is probably closely and similarly related to both observed sequences. If the two MINS values are similar and high, then the common ancestor is probably distantly and similarly related to both observed sequences. If mins(B→C) is low and mins(C→B) is high, then the common ancestor is probably more closely related to sequence C. And, if mins(B→C) is high and mins(C→B) is low, then the common ancestor is probably more closely related to sequence B.
I now felt prepared to meet again with Dr. Seppi and Dr. Clement about my biology-related PhD research plans.
Today Friday Feb 15 is the deadline for me to schedule my defense and graduate at the end of this semester. We met today and he gave me some feedback, but too late for me to respond. I felt that he was not fully engaged in helping me to finish in a timely manner.
The next day, I made some edits, responding to his feedback.
On Sunday, I felt strongly impressed to take corrective action, rather than continue on my current course. In the following days I did so, receiving specific instruction from the Spirit to guide me.
On March 20, Dr. Seppi became my advisor. Nine days later my thesis defense was scheduled, and on April 15 it was successfully defended. On the day of my defense, our UAV paper was accepted for publication. One week later, our application for Phase 2 UAV research funding was approved.
The first order of business was to put together a plan for a set of papers I would write, which papers would collectively serve as my dissertation. Originally, I wanted to show how MINS could be applied in various fields of study, including biology, but that plan was not approved. Ultimately, it was determined that my dissertation would focus on UAVs, of which applying MINS would be a part.
I dove right in to the task of writing a journal version of our UAV paper. I did that pretty much on my own, and it was accepted!
I also started work on a paper to introduce MINS as a relative measure of entropy to the machine learning community. I was really hoping to succeed with that paper, but after two tries I ran out of time. One reviewer complained that MINS was too simple to be considered for the important publication to which I had submitted it.
On September 3, 2013, in an NLP lab meeting, one of my fellow students said something that struck me. He said that the probability that a continuous random variable takes on a particular value is zero. In his mind this was simply an established fact, though a little difficult to fully understand. In my mind it was a contradiction. By definition, zero probability means that the event cannot happen. Immediately, I began to try to persuade my lab mates that someone had made a mistake. But that was a seemingly impossible task. An entire academic community accepted this apparent contradiction. And my friends were at school to get a PhD, not to try to prove others wrong who already had one.
I was immediately interested in discovering the root cause of the problem. Perhaps it was boyish curiosity. I started searching. After a month and a half I was led to axiomatic set theory, and to the definition of the word "all".
Meanwhile, there was plenty of work to be done participating in meetings, writing code, and running experiments for the Phase 2 UAV research project of which I was a part. One of the things we wanted to do was to integrate target tracking technology from the MAGICC lab called R-RANSAC with a probabilistic model for discrete sequence data used in our UAV paper called the Sequence Memoizer.
On March 21, 2014, I discovered the writings of Ludvig Wittgenstein from the Set Theory article in Wikipedia. I checked out a book written by him from the library and studied it. He complained about what was called the axiom of infinity, in axiomatic set theory. He also talked about difficulty in pinning down the meaning of the word "all". He pointed out that so-called completed infinite mathematical extensions are a contradiction-in-terms.
In December I caused our UAV research team to withdraw a submitted paper to avoid what I perceived as unfair competition, which was painful.
Also in December, I decided to eventually publish an extra-curricular paper in a field called philosophy of mathematics. My name for the paper while working on it was simply my def paper (definition paper), because I felt that clear definitions were needed.
By February 10, 2015, I was thinking about what it meant to define a domain of discourse in ZFC, prevailing axiomatic set theory.
On February 14, I had the idea that ZFC is inconsistent regardless of interpretation, and while serving in the temple later that day had an idea for writing a proof. The next week while struggling to write the proof I realized that the axiom of pairing was inconsistent; it was the problem. The axiom of infinity relied upon the domain of discourse which supposedly had been defined as infinite because of the axiom of pairing. But definition of the domain of discourse was never specified to have been completed. That definition wasn't finished, and therefore was not defined!
I remembered something that Steven R. Covey taught, that leadership is a choice. I decided that I would not seek employment related to UAVs, but rather some sort of employment related to my def paper. I wanted to finish my PhD as soon as possible.
On Saturday February 28 I submitted my paper to Robert Thomas at Philosophia Mathematica. He responded immediately that I had sent it to the wrong journal, and suggested that I send it to the Journal of Symbolic Logic instead, which I did on Monday. The paper was not long, and did not take long to write, just three pages.
The editor, Philip Welch, also responded immediately, and wrote:
"ZF is an impredicative theory and hence allows for (indeed requires) an infinite domain of discourse to which your arguments do not apply."
ZFC was used as the definition of an infinite domain of discourse. I don't think he thought very deeply about my paper. I believed that no amount of explaining or revising would ever convince him to consider it seriously.
On Wednesday I put the paper on my home page, on our lab's computer, and on Thursday I invited four leading math professors to look at it. Hugh Woodin at Harvard was kind enough to do so and sent the following reply:
"Your argument seems to assume that the domain (of discourse) is finite."
Within an hour I wrote back:
"It does seem that way, but the argument is for any given domain of discourse."
He didn't respond further. I really appreciated his willingness to read and talk with me about my paper. I learned a lot from our brief exchange. A week later I had another longer exchange with another Harvard professor which helped me to understand why my paper was difficult to understand. It was because people did not realize that they were using the term "finite" to mean "defined" or "given". As soon as I realized this I wrote back to Hugh Woodin and explained:
"I feel that I can give a more direct response to your comment that my argument seems to assume that the domain of discourse is finite. My argument assumes that the domain of discourse is given or defined. Perhaps the word finite means given or defined."
I was a bit sharp, writing that perhaps the word finite means given or defined. Because, of course, if that is the case then the word infinite means not given or not defined. I did not mean to be unkind or rude, just clear.
I also wrote back to Philip Welch explaining the same point:
"Please permit me a brief response. My arguments apply to a given or defined domain of discourse. Perhaps that is what is meant generally by the term finite. Thank you."
And to the other Harvard professor I wrote:
"When you said ZFC has no finite model, I should have responded that ZFC has no given or defined model, which is why it is inconsistent."
After Harvard, I wrote to other full professors at Berkeley, UCLA, Princeton, and Oxford. I greatly appreciated those who were willing to read my paper, and to tell me what they thought about it. Academics have been doing this sort of thing for a long time, writing down their ideas and asking others to read what they have written. I put myself in a position to be corrected by those who should be best able to teach me. I listened carefully to all those who were kind enough to take the time to communicate with me.
On April 15 I received the email I wrote about at the beginning of this story from someone I had not contacted, Michael Beeson. I did not perceive his communication as kind, rather I positively felt threatened. About two weeks later I was writing Jeremy Kahn from CUNY about my paper and he asked me if I had talked with any of the professors in my department about it. I hadn't, but I didn't want to tell him that. I knew that my professors wouldn't want to have anything to do with it. I felt like a lamb having a casual chat with a wolf in a dark corner of the woods. For some reason, I decided to be trusting and answered honestly that I had not. Three days later my advisor told me that he spoke for himself and the department by demanding that I stop contacting people about my paper.
My paper was no longer merely academic. It became political.
I didn't know why, but after my advisor had delivered his message, I received clear instruction from the Spirit to refuse to comply with any demands. The next day, a Saturday, I sent an email refusing to comply. That was difficult for me to do. I guess I was getting used to difficult things. It was very difficult for me to follow instruction from the Spirit which led to Dr. Seppi becoming my advisor two years earlier. It is difficult for me to write this story.
I continued faithful following that instruction until new instruction came, which came the day after my computer access was disabled. This new instruction was to stop sending emails as I had been doing, and very importantly NOT to agree that I would stop sending them. It so happened that not agreeing to stop contacting people about my paper would also be very difficult! It was not difficult to stop sending the emails.
It was hard to have people at BYU upset with me, and thinking critically of me. I didn't like it at all. And it was super distressing thinking that I might be kicked out of the university.
However, I understood that the pressure came from people outside of BYU. Those people were simply abusing my school.
I hope that by reading this story you come to understand that there is competition in the academic world between genuine learning and protecting reputation.
I also hope you believe what I have said about the role that the Spirit of the Lord has had in my adventures in graduate school. I have told the truth.
I know that God lives and that he is able and willing to help each of us learn, especially when we are willing to hear Him.
THE END
Last revised 24 June 2023
Copyright © 2023 Kevin Cook