David's life has made more than one movie, and the story of his rise from shepherd boy to warrior to king certainly catches people's attention.
But if we were to make such a movie, we might have a hard time putting this scene in for modern audiences. Saul has been the villain of the first part of the story. Although the king brought David into court so that the younger man's musical skills could ease his depression, things have soured. David's skill in battle gained him greater fame than Saul, and people sing greater praises of him. Though Saul married David to one of his daughters, he has also tried to kill him and drove him first out of the palace and then out of the country. This is considered extreme behavior even for fathers-in-law.
David has been on the run for years and a few times has been cornered by Saul, to escape only at the last moment. Saul, as the king, stands in the way of David's assuming the kingship that was promised to him when Samuel anointed him. So Saul's death solves a number of problems if this were just a dramatic story -- it allows David to return home and it allows him to make his move on the throne itself.
Instead David responds not with triumph or gratitude that Saul is dead but grief. We get how he would grieve the death of his friend Jonathan, but why would he be so sad about the death of a man who'd spent the last several years trying to kill him?
We get clues when we read the lament song that David wrote. Firstly, Israel's enemies will rejoice at Saul's death. For all his problems, Saul was a great warrior and an effective military leader. Nations that wanted to defeat Israel in battle just got a good head start. David may know he was supposed to be king, but Saul's remaining sons might have the idea they're supposed to follow their father on the throne, which means internal unrest and turmoil -- maybe even civil war.
Saul's success on the battlefield brought wealth and plunder to Israel. In many ways, he was a good king, and his inability to obey God didn't seem to harm the rest of the people of the nation very much. So David grieves the loss of a great warrior and effective leader for his nation.
I wonder if David also doesn't lament the loss of what Saul could have been. Had he obeyed God he could have been established as the founder of a great dynasty. Even after his disobedience, he could have turned his life around, maybe resigned as king and stayed on as David's adviser. His experience and knowledge could have been useful to a man who'd never been a king before. Saul's advice could have made David a better king, which would have been better for the nation of Israel.
But his death means he won't have the chance to redeem himself or to ask God to forgive his disobedience.
You might come up with other reasons if you think about it, but I think you might find that most of them are similar to the ones I mention in at least one important respect: They're about something bigger than David himself. Yes, I believe that David understood Saul's death made much of his life easier. But he sees a bigger picture, and he realizes that whatever benefits Saul's death provides him, it provides more problems for the people as a whole.
Jesus asks us to see beyond ourselves when we follow him. He may call us to spend more time, energy and money on others that we would rather spend on ourselves. He may call us to consider what other people need when we only want to consider what we want. He himself saw beyond what his own desires might have been -- because whatever they were, it's a safe bet they didn't include "getting beaten up and nailed to a tree" -- in order to do something for others, and he asks us to do the same.
Sure, few of are likely to be called to that extreme level of sacrifice, but almost all of us are called to some. We are called to see, like David, that sometimes what looks good for us may not look good in the bigger picture and trust that God, who sees a bigger picture still, has in mind what is ultimately right and best for all.